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ABSTRACT 



In 1990, the National Assessment of Educational Progress 
(NAEP) included a Trial State Assessment (TSA) ; for the first time in the 
NAEP’s history, voluntary state-by-state assessments were made. The sample 
was designed to represent the 8th grade public school population in a state 
or territory. In 1996, 44 states, the District of Columbia, Guam, and the 
Department of Defense schools, took part in the NAEP state science assessment 
program. The NAEP 1996 state science assessment was at grade 8 only, although 
grades 4, 8, and 12 were assessed at the national level as usual. The 1996 
state science assessment covered three major fields: earth, physical, and 
life sciences. In New York, 1,876 public school and 514 nonpublic school 
students in 82 public schools and 28 nonpublic schools were assessed. This 
report describes the science proficiency of New York eighth-graders, compares 
their overall performance to students in the Northeast region of the United 
States and the entire United States (using data from the NAEP national 
assessment) , presents the average proficiency for the three major fields, and 
summarizes the performance of subpopulations (gender, race/ethnicity, 
parents' educational level, Title I participation, and free/reduced lunch 
program eligibility) . To provide a context for the assessment data, 
participating students, their science teachers, and principals completed 
questionnaires which focused on: instructional content (curriculum coverage, 
amount of homework) ; delivery of science instruction (availability of 
resources, type) ; use of computers in science instruction; educational 
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background of teachers; and conditions facilitating science learning (e.g., 
hours of television watched, absenteeism) . On the NAEP fields of science 
scales that range from 0 to 300, New York students had an average proficiency 
of 146 compared to 148 throughout the United States. The average science 
scale score of males did not differ significantly from that of females in 
either New York or the nation. At the eighth grade. White students in New 
York had an average science scale score that was higher than those of Black 
and Hispanic students but was not significantly different from that of 
Asian/Pacific Islander students. In New York at grade 8, the average scale 
score of public students (146) was not significantly different from that of 
nonpublic school students (149) . (DDR/NB) 
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HIGHLIGHTS 



^^[onitoring the performance of students in subjects such as science is a key 
concern of the citizens, policy makers, and educators who direct educational reform 
efforts. The 1996 National Assessment of Educational Progress (NAEP) in science 
assesses the current level of science performance as a mechanism for informing 
education reform. This science assessment is the first to be constructed on a new 
framework, and it is also the first to be given at the state level. This report contains 
results for public and nonpublic school students at grade 8. 

What Is NAEP? 

The National Assessment of Educational Progress (NAEP), the “Nation’s Report 
Card,” is the only ongoing nationally representative assessment of what America’s 
students know and can do in various academic subjects. Since 1969, NAEP assessments 
have been conducted with national samples of students in the areas of reading, 
mathematics, science, writing, and other fields. By making information on student 
performance available to policy makers, educators, and the general public, NAEP is an 
integral part of our nation’s evaluation of the conditions and progress of education. 

NAEP is a congressionally mandated project of the National Center for Education 
Statistics, U.S. Department of Education. Results are provided only for group 
performance. NAEP is forbidden by law to report results at an individual or school 
level. 

In 1990 Congress authorized a voluntary state-by-state NAEP assessment. The 
r 1990 Trial State Assessment in mathematics at grade 8 was the first state-level NAEP 
assessment. Since then, state-level assessments have taken place in 1992 and 1994 in 
reading (grade 4), in 1992 and 1996 in mathematics (grades 4 and 8), and in 1996 in 
science (grade 8). In 1996, 44 states, the District of Columbia, Guam, and the 
Department of Defense Schools took part in the NAEP state assessment program. The 
NAEP 1996 state science assessment was at grade 8 only, although grades 4, 8, and 12 
were assessed at the national level as usual. 
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NAEP 1996 Science Assessment 

The NAEP 1996 science assessment was developed using a new framework. This 
framework was produced by educators, administrators, assessment experts, and 
curriculum specialists using a national consensus process. The framework was designed 
to reflect current practices in science teaching. It called for the use of multiple-choice 
questions and constructed-response questions that required both short and extended 
responses. The constructed-response questions served as indicators of students’ ability 
to know and integrate facts and scientific concepts, their ability to reason, and their 
ability to communicate scientific information. In the 1996 assessment, these 
constructed-response questions constituted nearly 80 percent of the total student response 
time. The NAEP 1996 assessment in science also included hands-on tasks that enabled 
students to demonstrate directly their knowledge and skills related to scientific 
investigation. 

The 1996 science framework was structured according to a matrix that consisted 
of the three traditional fields of science (earth, physical, and life) crossed with three 
processes of knowing and doing science (conceptual understanding, scientific 
investigation, and practical reasoning). A central category encompassing the nature of 
science and the nature of technology was woven throughout the assessment, as was a 
themes category representing major ideas or key concepts that transcend scientific 
disciplines. 1 

Students’ science performance is summarized on the NAEP science scales, which 
range from 0 to 300 at each grade. While the scale score ranges are identical for grades 
4, 8, and 12, the scales were derived independently at each grade. For example, scale 
scores on the grade 8 scale cannot imply anything about performance at grade 12 in the 
national assessment. The science scale is discussed in Appendix C of this report, the 
NAEP 1996 Science State Report for New York (see C.9). Note that the national average 
for the combined public and nonpublic school population is 150; the average for public 
schools only (appropriate for most tables in this report) is- 148. 

Comparison of New York to the Nation 

Table H.1 shows the distribution of science scale scores for eighth-grade students 
attending public schools in New York, the Northeast region, and the nation in 1996. 
See Chapter 2 (Table 2.6) of this report for the results for the nonpublic and the 
combined public and nonpublic school populations. 

• The average science scale score for eighth graders in public schools in 
New York was 146. This average did not differ significantly from that 
for public school students across the nation (148). 2 




1 More details about the NAEP 1996 science assessment can be found in Appendix B of this report, the NAEP 1996 
Science State Report for New York. 

2 Differences reported as significant are statistically different at the 95 percent confidence level. This means that with 
95 percent confidence there is a real difference in the average science scale score between the two populations of 
interest. 
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THE NATION'S 

i 

State Assessment 


TABLE H.1 


Distribution of Science Scale Scores for Public School 
Students at Grade 8 





Average 
Scale Score 


10th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


90th 

Percentile 
















New York 
Northeast 
Nation 


146(1.6) 
149 ( 2.9) 
148 ( 0.9) 


96 ( 2.9) 
103 ( 4.8) 
102 ( 1.6) 


122 ( 2.5) 
127 ( 4.5) 
126 ( 1.3) 


149 ( 2.0) 
151 ( 3.0) 
151 (0.9) 


172 ( 1.4) 

173 ( 4.0) 
172 ( 1.1) 


190 ( 1.3) 

191 ( 3.0) 
191 ( 1.3) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



Major Findings for Student Subpopulations 

The preceding section provided a view of the overall science performance of 
eighth-grade students in New York. It is also important to examine the average science 
scale scores of subgroups within the population. Typically, NAEP presents results for 
demographic subgroups defined by gender, race/ethnicity, and parental education. In 
addition, in 1996 NAEP collected information on student participation in two federally 
funded programs: Title I programs and the free/reduced-price lunch component of the 
National School Lunch Program. The 1996 state assessment in science also continues 
a component first introduced with the NAEP 1994 state assessment in reading — 
assessment of a representative sample of nonpublic school students. 

The reader is cautioned against using NAEP results to make simple or causal 
inferences related to subgroup membership. Differences among groups of students are 
almost certainly associated with a broad range of socioeconomic and educational factors 
not discussed in NAEP reports and possibly not addressed by the NAEP assessment 
program. 
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Results related to gender and race/ethnicity for public school students are 
highlighted below. A comparison of public and nonpublic school results is also 
presented. More complete results for the various demographic subgroups examined by 
the NAEP science assessment can be found in Chapter 2 of this report, the NAEP 1996 
Science State Report for New York. 

• The average science scale score of males did not differ significantly 
from that of females in either New York or the nation. 

• At the eighth grade, White students in New York had an average science 
scale score that was higher than those of Black and Hispanic students 
but was not significantly different from that of Asian/Pacific Islander 
students. 

• In New York at grade 8, the average scale score of public school 
students (146) was not significantly different from that of nonpublic 
school students (149). 

Finding a Context for Understanding Students’ Science Performance 
in Public Schools 

The science performance of students in New York may be better understood when 
viewed in the context of the environment in which students are learning. This 
educational environment is largely determined by school policies and practices, by 
characteristics of science instruction in the school, by home support for academics and 
other home influences, and by students’ own views about science. Information about 
this environment is gathered by means of questionnaires completed by principals and 
teachers as well as questions answered by students as part of the assessment. 

Because NAEP is administered to a sample of students that is representative of 
all eighth-grade students in New York schools, NAEP results provide a view of the 
educational practices in New York that may be useful for improving instruction and 
setting policy. However, despite the richness of context provided by the NAEP results, 
it is very important to note that NAEP data cannot establish a cause-and-effect 
relationship between educational environment and students’ scores on the NAEP science 
assessment. 
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The following results are for public school students: 

School Science Education Policies and Practices 3 

• In New York, the percentage of eighth-grade students attending public 
schools that reported science was a priority (49 percent) was not 
significantly different from the percentage of eighth-grade students 
nationwide (43 percent). 

• The percentage of eighth-grade students in New York who attended 
schools that were expected to follow a district or state curriculum 
(93 percent) was not significantly different from the national percentage 
(94 percent). 

• In New York, 94 percent of eighth graders attended schools that 
reported providing instruction in science every day. This percentage did 
not differ significantly from that of eighth graders across the nation 
(92 percent). 

• Relatively few of the students in New York had- teachers who reported 
receiving all of the resources they needed for classroom instruction 
(11 percent). This was not significantly different from the 
corresponding percentage of eighth-grade students nationwide 
(11 percent). 

• In New York, 35 percent of the eighth-grade students were taught by 
teachers who reported that there was a curriculum specialist available 
to help or advise them in science. This figure did not differ significantly 
from* that of students across the nation (43 percent). 

Science Classroom Practices 4 

• Less than half of the eighth-grade students in New York had science 
teachers who reported spending a lot of time on earth science 
(38 percent), about three quarters reported spending a lot of time on 
physical science (73 percent), and less than one fifth reported spending 
a lot of time on life science (17 percent). 

• Relatively few of the students in New York (10 percent) had teachers 
who reported they planned to place moderate emphasis on the 
understanding of key science concepts by their students. This 
percentage was smaller than that of students whose teachers planned 
heavy emphasis on conceptual understanding (90 percent). 



* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 

3 

More detailed results related to school policies and practices can be found in Chapter 3 of this report, the NAEP 1996 
Science State Report for New York. 

4 

More detailed results related to classroom practices can be found in Chapter 4 of this report, the NAEP 1996 Science 
State Report for New York. 
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• In New York, the percentage of eighth-grade students whose teachers 
reported they planned to give moderate emphasis to developing science 
problem-solving skills (33 percent) was smaller than that of students 
whose teachers planned heavy emphasis on this topic (62 percent). 

• Teachers of 56 percent of the students in New York reported that they 
planned to place moderate emphasis on knowing how to communicate 
ideas in science effectively, greater than the percentage of students 
whose teachers reported giving this topic heavy emphasis (35 percent). 

• In New York, 10 percent of eighth graders reported not spending any 
time on science homework in a typical week while 43 percent spent one 
hour or more on their science homework each week. 

Scientific Investigations 5 

• Of the eighth-grade students in New York, 88 percent had teachers who 
reported giving moderate to heavy emphasis on the development of data 
analysis skills. This percentage was not significantly different from that 
of students nationwide (89 percent). 

• About two thirds of the eighth graders in New York had teachers who 
reported their students performed hands-on activities or investigations 
in science once a week or more (70 percent). 

Influences Beyond School That Facilitate L ear ning Science 6 

• The percentage of eighth graders in New York who reported watching 
six or more hours of television a day (16 percent) was not significantly 
different from the percentage for the nation (17 percent). 

• In New York, 40 percent of eighth graders agreed that science is useful 
for solving everyday problems. 




5 More detailed results related to scientific investigations can be found in Chapter 5 of this report, the NAEP 1996 Science 
State Report for New York. 

6 More detailed results related to influences beyonds school that facilitate learning science can be found in Chapter 6 of 
this report, the NAEP 1996 Science State Report for New York. 
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INTRODUCTION 



T 

improving education is often seen as an important first step as the United States 
attempts to remain competitive in an increasingly technical global economy. At the 1996 
Governors’ Summit in Palisades, New Jersey, the President and the Governors 
reaffirmed the need to strengthen our schools and strive for world-class standards. 
Furthermore, in his 1997 State of the Union Address, President Clinton placed education 
center stage and called for states to commit to national standards that represent what all 
students must know to succeed in the knowledge-based economy of the twenty-first 
century. 

In 1983, the National Commission on Excellence in Education issued a report 
entitled A Nation at Risk : The Imperative for Educational Reform that was critical of 
education in the United States. 7 Interest in reform was also fueled by the publication 
of other reports and analyses that pointed out the deficiencies of the educational system 
and noted how these could be rectified. 8 Since then, organizations from the public and 
private sectors have assumed pivotal roles in providing support to state and local 
educational establishments as they seek to reform their educational systems in areas such 
as the development of standards, revision of curricula, development of appropriate 
assessment techniques, and professional development 9 In addition to these activities, 
organizations such as the National Science Teachers Association and the American 
Association for the Advancement of Science have worked closely with the National 
Research Council to produce documents that help teachers interpret the National Science 
Education Standards that were published in 1995. 10 As the new century approaches, 
commitment to science reform continues. 



7 A Nation at Risk: The Imperative for Educational Reform . (Washington, DC: National Commission on Excellence in 
Education, 1983). 

g 

Educating Americans for the 21st Century: A Report to the American People and the National Science Board . 
(Washington, DC: National Science Board, Commission on Precollege Education in Mathematics, Science, and 
Technology, 1983). 

9 

Statewide Systemic Initiatives in Science , Mathematics , and Engineering . (Arlington, VA: The National Science 
Foun dati on, 1995-1996); Scope , Sequence, and Coordination of Secondary School Science. Volume I: The Content Core ; 
Volume II: Relevant Research. (Washington, DC: National Science Teachers Association, 1992); Benchmarks for 
Science Literacy. (Washington, DC: Project 2061, American Association for the Advancement of Science, 1993); New 
Standards Project. (Washington, DC: National Research Council, 1995). 

10 National Science Education Standards. (Washington, DC: National Research Council, 1996). 
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Monitoring the performance of students in science is a key concern of the state 
and national policy makers and educators who direct educational reform efforts. To this 
end, the 1996 National Assessment of Educational Progress (NAEP) is an important 
source of information on what the nation’s students know and can do in science. 

What Was Assessed? 

The science assessment was crafted to measure the content and skills specified in 
the science framework for the 1996 NAEP. Two organizing concepts underlie the 
science framework. First, scientific knowledge should be structured so as to make 
factual information meaningful. The way in which knowledge is structured should be 
influenced by the context in which the knowledge is being presented. Second, science 
performance depends on knowledge of facts, the ability to integrate this knowledge into 
larger constructs, and the capacity to use the tools, procedures, and reasoning processes 
of science to develop an increased understanding of the natural world. Thus, the 
framework called for the NAEP 1996 science assessment to include the following: 

• Multiple-choice questions that assess students’ knowledge of important 
facts and concepts and that probe their analytical reasoning skills; 

• Constructed-response questions that explore students’ abilities to 
explain, integrate, apply, reason about, plan, design, evaluate, and 
communicate scientific information; and 

• Hands-on tasks that probe students’ abilities to use materials to make 
observations, perform investigations, evaluate experimental results, and 
apply problem-solving skills. 

The core of the science framework is organized along two dimensions. The first 
dimension divides science into three major fields: earth, physical, and life sciences. 

The second dimension defines characteristic elements of knowing and doing science: 
conceptual understanding, scientific investigation, and practical reasoning. Each 
question in the assessment is categorized as measuring one of the elements of knowing 
and doing within one of the fields of science (e.g., scientific investigation in the context 
of earth science). The framework also contains two overarching domains — the nature 
of science and the organizing themes of science. The nature of science encompasses the 
historical development of science and technology, the habits of mind that characterize 
science, and the methods of inquiry and problem solving. It also includes the nature 
of technology — specifically, design issues involving the application of science to 
real-world problems and associated trade-offs or compromises. The themes of science 
include the notions of systems and their application in the scientific disciplines, models 
and their functioning in the development of scientific understanding, and patterns of 
change as they are exemplified in natural phenomena. A fuller description of the 
framework is provided in Appendix B. 
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Who Was Assessed? 

School and Student Characteristics 

Table 1.1 provides demographic profiles of the eighth-grade students in New York, 
the Northeast region, and the nation. These profiles are based on data collected from 
the students and schools participating in the 1996 state and national science assessments 
at grade 8. As described in Appendix A, the state data and the regional and national 
data are drawn from separate samples. 

In 1996, approximately 84 percent of eighth graders in New York attended public 
schools, with the remaining students attending nonpublic schools (including Catholic and 
other private schools). For the nation, 89 percent of students at grade 8 attended public 
schools in 1996. 

To ensure comparability across jurisdictions, NCES has established guidelines for 
school and student participation rates. Appendix A highlights these guidelines, and 
jurisdictions failing to meet these guidelines are noted in tables and figures in N AF.P 
reports containing state-by-state results. For jurisdictions failing to meet the initial 
school participation rate of 70 percent, results are not reported. 

Schools and Students Assessed 

Table 1.2 summarizes participation data for schools and students sampled in New 
York for the 1996 state assessment program in science. 11 

In New York, 82 public schools and 28 nonpublic schools participated in the 1996 
eighth-grade science assessment. These numbers include participating substitute schools 
that were selected to replace some of the nonparticipating schools from the original 
sample. The weighted school participation rate after substitution in 1996 was 78 percent 
for public schools and 87 percent for nonpublic schools, which means that the 
eighth-grade students in this sample were directly representative of 78 percent and 
87 percent of all the eighth-grade public and nonpublic school students, respectively, in 
New York. 

In each school, a random sample of students was selected to participate in the 
assessment. In New York in 1996, on the basis of sample estimates, 8 percent of the 
eighth-grade public school population and 0 percent of the nonpublic school population 
were classified as students with limited English proficiency (LEP). In addition, 

8 percent of eighth graders in public schools and 2 percent of eighth graders in 
nonpublic schools had an Individual Education Plan (IEP). An IEP is a plan written for 
a student who has been determined to be eligible for special education. The IEP 
typically sets forth goals and objectives for the student and describes a program of 
activities and/or related services necessary to achieve the goals and objectives. 




u 



For a detailed discussion of the NCES guidelines for sample participation, see Appendix A of this report or the 
Technical Report of the NAEP 1996 State Assessment Program in Science. (Washington, DC: National Center for 
Education Statistics, 1997). 
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TABLE 1.1 



Profile of Students in New York, the Northeast Region, and 
the Nation at Grade 8 



Demographic Subgroups 



Public 


Nonpublic 


Combined 


Percentage 



1 RA CE/ETHNICFTY 














New York 


White 


60 


(2.6) 


42 ( 7.6) 


57 


(2.4) 




Black 


17 


(2.0) 


34 ( 6.6) 


20 


(1.8) 




Hispanic 


16 


(1.2) 


18 ( 3.3) 


16 


(1.1) 




Asian/Pacific Islander 


5 


(0.9) 


4(1.0) 


5 


(0.8) 




American Indian 


2 


(0.5) 


2 ( 0.7) 


2 


(0.4) 


Northeast 


White 


62 


(5.5) 


79 ( 5.7) 


65 i 


(4.7) 




Black 


25 


(3.8) 


10 ( 3.3) 


22 i 


(2.9) 




Hispanic 


10 


(3.8) 


8 ( 4.6) 


9 i 


( 3.2) 




Asian/Pacific Islander 


2 i 


(0.8) 


3(1.4) 


2 i 


(0.7) 




American Indian 


2 i 


(0.5) 


0(—) 


1 i 


(0.4) 


Nation 


White 


68 i 


(0.4) 


80 ( 2.7) 


70 i 


(0.2) 




Black 


15 i 


(0.3) 


7(1.5) 


14 I 


(0.1) 




Hispanic 


12 i 


(0.3) 


9(2.1) 


12 1 


(0.2) 




Asian/Pacific Islander 


2 i 


(0.3) 


4 ( 0.8) 


3 1 


(0.3) 




American Indian 


2 i 


(0.3) 


1 (0.2) 


21 


(0.2) 


PARENTS' EDUCATION 












New York 


Did not finish high school 


6 1 


(0.7) 


3(1.0) 


51 


(0.6) 




Graduated from high school 


16 1 


(0.9) 


13 (2.7) 


15 I 


(0.9) 




Some education after high school 


19 1 


(1.1) 


15 (2.0) 


18 1 


(0.9) 




Graduated from college 


49 1 


(1.4) 


59 ( 5.4) 


51 I 


(1.4) 




1 don't know. 


11 1 


(0.8) 


10 ( 22) 


11 1 


(0.6) 


Northeast 


Did not finish high school 


5 I 


(1.4) 


1 ( 0.6) 


4 1 


(1.1) 




Graduated from high school 


23 1 


( 3.6) 


10 ( 2.1) 


20 1 


(2.8) 




Some education after high school 


19 1 


(1.3) 


15 ( 3.8) 


18 1 


(1.2) 




Graduated from college 


42 1 


(2.4) 


65 ( 6.4) 


47 1 


(2.4) 




1 don't know. 


10 | 


(2.4) 


9(1.6) 


10 1 


(1.8) 


Nation 


Did not finish high school 


7 1 


(0.5) 


2 ( 0.3) 


61 


(0.4) 




Graduated from high school 


21 1 


(1.0) 


10(1.1) 


20 1 


(0.9) 




Some education after high school 


20 1 


(0.7) 


17(1.8) 


20 1 


(0.7) 




Graduated from college 


42 1 


(1.3) 


66 ( 3.0) 


45 1 


(1.2) 




1 don't know. 


10 I 


(0.6) 


6(1.0) 


9 1 


(0.5) 


GENDER 














New York 


Male 


50 I 


(1.0) 


51 (6.1) 


51 I 


(1.3) 




Female 


50 1 


(1.0) 


49(6.1) 


49 1 


(1.3) 


Northeast 


Male 


51 | 


(4.5) 


51 ( 3.6) 


51 1 


(3.5) 




Female 


49 1 


(4.5) 


49 ( 3.6) 


49 1 


(3.5) 


Nation 


Male 


51 1 


(1 2) 


51 ( 1 .8) 


51 1 


(1.0) 




Female 


49 1 


{^2) 


49 ( 1 .8) 


49 1 


(1.0) 



(continued on next page) 
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TABLE 1.1 (continued) 


Profile of Students in New York, the Northeast Region, and 
the Nation at Grade 8 



Demographic Subgroups 


Public 


Nonpublic 


Combined 


Percentage 




TITLE f 














New York 


Participated 


16 


(2.6) 


2 ( 1.2) 


14 


(2 2) 




Did not participate 


64 


(2.6) 


98 ( 1 .2) 


86 


(2.2) 


Northeast 


Participated 


15 


(8.5) 


14(7.1) 


15 


(7.0) 




Did not participate 


85 


(8.5) 


86 ( 7.1) 


85 


(7.0) 


Nation 


Participated 


13 


(2.3) 


7 ( 3.6) 


12 


(2.1) 




Did not participate 


87 


(2.3) 


93 ( 3.6) 


68 


(2.1) 


FREE/REDUCED-PRICE LUNCH 












New York 


Bigible 


37 


(2.3) 


19 ( 7.0) 


34 


(2 2) 




Not eligible 


54 


(2.8) 


60 ( 9.1) 


55 


(2.8) 




Information not available 


9 


(2.6) 


20 ( 7.0) 


11 


(2.5) 


Northeast 


Eligible 


43 


(3.0) 


12 ( 7.0) 


37 


( 3.2) 




Not eligible 


56 


(3.2) 


47 (15.2) 


54 


( 4.2) 




Information not available 


1 


(0.4) 


42 (16.3) 


9 


(3.9) 


Nation 


Eligible 


29 


(1.6) 


7 ( 3.4) 


26 


(1.5) 




Not eligible 


51 


(3.6) 


49 ( 7.7) 


51 


(3.3) 




Information not available 


20 


(4.4) 


44 ( 8.2) 


23 


(4.1) 



The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, for each 
population of interest, the value for the entire population is within ± 2 standard errors of the estimate for the sample. In 
comparing two estimates, one must use the standard error of the difference (see Appendix A for details). The percentages 
for Race/Ethnicity may not add to 100 percent because some students categorized themselves as “Other.” 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 



Schools were permitted to exclude certain students from the assessment, provided 
that the following criteria were met. To be excluded, a student had to be categorized 
as LEP or had to have an IEP and (in either case) be judged incapable of participating 
in the assessment. The intent was to assess all selected students; therefore, all selected 
students who were capable of participating in the assessment should have been assessed. 
However, schools were allowed to exclude those students who, in the judgment of school 
staff, could not meaningfully participate. The NAEP guidelines for inclusion are 
intended to assure uniformity of inclusion criteria from school to school. Note that some 
students classified as LEP and some students having an EEP were deemed eligible to 
participate and were included in the assessment. In New York, the students who were 
excluded from the assessment because they were categorized as LEP or had an IEP 
represented 9 percent of the public school population and 1 percent of the nonpublic 
school population in grade 8. 




New York 



In New York, 1,876 public school and 514 nonpublic school eighth-grade students 
were assessed in 1996. The weighted student participation rate was 90 percent for 
public schools and 97 percent for nonpublic schools. This means that the sample of 
eighth-grade students who took part in the assessment was directly representative of 
90 percent of the eligible public school student population and 97 percent of the eligible 
nonpublic school population in participating schools in New York (that is, all students 
from the population represented by the participating schools, minus those students 
excluded from the assessment). The overall weighted response rate (school rate times 
student rate) was 70 percent and 85 percent for public and nonpublic schools, 
respectively. This means that the sample of students who participated in the assessment 
was directly representative of 70 percent of the eligible eighth-grade public school 
population and 85 percent of the eligible eighth-grade nonpublic school population in 
New York. 

In accordance with standard practice in survey research, the results presented in 
this report were based on calculations that incorporate adjustments for the 
nonparticipating schools and students. Hence, the final results derived from the sample 
provide estimates of the science performance for the full population of eligible public 
and nonpublic school eighth-grade students in New York. However, in instances where 
nonparticipation rates are large, these nonparticipation adjustments may not adequately 
compensate for the missing sample schools and students. 

In order to guard against potential nonparticipation bias in published results, the 
National Center for Education Statistics (NCES) has established minimum participation 
levels as a condition for the publication of 1996 state assessment program results. NCES 
also established additional guidelines addressing four ways in which nonparticipation 
bias could be introduced into a jurisdiction’s published results (see Appendix A). In 
1996 New York met minimum participation levels for both public and nonpublic schools 
at grade 8. Hence, results for both types of schools are included in this report. New 
York failed to meet other established NCES participation guidelines. The public and 
nonpublic school weighted participation rates for the initial samples of schools were 
below 85% AND the weighted school participation rates after substitution were below 
90% (see Appendix A). 

In the analysis of student data and reporting of results, nonresponse weighting 
adjustments have been made at both the school and student level, with the aim of making 
the sample of participating students as representative as possible of the entire eligible 
eighth-grade population. For details of the nonresponse weighting adjustment 
procedures, see the Technical Report of the NAEP 1996 State Assessment Program in 
Science. 
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TABLE 1.2 



School and Student Participation at Grade 8 in New York 





Public 


Nonpublic 


SCHOOL PARTICIPATION 






Weighted school participation rate before substitution 


70% 


84% 


Weighted school participation rate after substitution 


78% 


87% 


Number of schools originally sampled 


106 


39 


Number of schools not eligible 


0 


6 


Number of schools in original sample participating 


74 


27 


Number of substitute schools provided 


30 


5 


Number of substitute schools participating 


8 


1 


Total number of participating schools 


82 


28 


STUDENT PARTICIPATION 






Weighted student participation rate after makeups 


90% 


97% 


Number of students selected to participate in the 






assessment 


2,222 


537 


Number of students withdrawn from the assessment 


41 


4 


Percentage of students who were of Limited English 






Proficiency 


8% 


0% 


Percentage of students excluded from the assessment 






due to Limited English Proficiency 


5% 


0% 


Percentage of students who had an Individualized 






Education Plan 


8% 


2% 


Percentage of students excluded from the assessment 






due to Individualized Education Plan status 


4% 


1% 


Number of students to be assessed 


2,068 


530 


Number of students assessed 


1,876 


514 


Overall weighted response rate 


70% 


85% 



New York’s public and nonpublic school weighted participation rates for the initial samples of schools were . below 85% 
AND the weighted school participation rates after substitution were below 90%. See Appendix A for details. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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Reporting NAEP Science Results 

The NAEP Science Scale 

The NAEP 1996 science assessment spans the broad field of science in each of 
the grades assessed. Because of the survey nature of the assessment and the breadth 
of the domain, each student participating cannot be expected to answer all the questions 
in the assessment since this would impose an unreasonable burden on students and their 
schools. Thus, each student was administered a portion of the assessment, and data were 
combined across students to report on the achievement of eighth graders and on the 
achievement of subgroups of students (e.g., subgroups defined by gender or parental 
education). 

Student responses to the assessment questions were analyzed to determine the 
percentage of students responding correctly to each multiple-choice question and the 
percentage of students achieving each of the score categories for constructed-response 
questions. Item response theory (IRT) methods were used to produce scales that 
summarized results for each of the three fields of science (i.e., earth, physical, and life) 
at each grade level. An overall composite scale also was developed at each of grades 
4, 8, and 12 by weighting the separate scales based on the relative importance of each 
field of science in the NAEP science framework. Results presented in this report are 
based on this overall composite scale, which ranges from 0 to 300. 

The use of separate grade-specific reporting scales for the science assessment is 
consistent with the National Assessment Governing Board’s 1993 policy that future 
NAEP assessments be developed using within-grade frameworks and that scaling be 
carried out within grade. Because this science assessment was based on a new 
framework, and no comparisons with previous NAEP science assessments were possible, 
a new scale was developed. The ranges of the science scales (from 0 to 300) differ by 
design from the 0-to-500 reporting scales used in other NAEP subject areas and were 
chosen to minimize confusion with other common test scales and to discourage 
inappropriate cross-grade comparisons. 

The national average on the science scale is 150, including both public and 
nonpublic school students. The average for the nation’s public school students appears 
most frequently in this report, and it is slightly lower. (Additional details of the scaling 
procedures can be found in Appendix C of this report, in the NAEP 1996 Technical 
Report , and in the Technical Report of the NAEP 1996 State Assessment Program in 
Science .) 





14 



THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



New York 



Science Achievement Levels 

A companion report, being issued by the National Assessment Governing Board, 
will present the NAEP 1996 science results in terms of achievement levels. As 
authorized by the NAEP legislation and adopted by the National Assessment Governing 
Board, the achievement levels are based on the Board’s judgments about what are 
reasonable performance expectations for students on the NAEP 1996 science assessment. 
The achievement levels for the NAEP 1996 science assessment were adopted on an 
interim basis, indicating that they may be revised when other information becomes 
available, such as the fourth- and twelfth-grade results from the Third International 
Mathematics and Science Study (TIMSS). 

Interpreting NAEP Results 

This report describes science performance for eighth graders and compares the 
results for various groups of students within that population — for example, those who 
have certain demographic characteristics or who responded to a specific background 
question in a particular way. The report examines the results for individual demographic 
groups and for individual background questions. It does not include an analysis of the 
relationships among combinations of these subpopulations or background questions. 

Because the percentages of students in these subpopulations and their average 
science scale scores are based on samples, rather than on the entire population of eighth 
graders in a jurisdiction, the numbers reported are necessarily estimates . As such, they 
are subject to a measure of uncertainty, reflected in the standard error of the estimate. 
When the percentages or average scale scores of certain groups are compared, it is 
essential to take the standard error into account, rather than to rely solely on observed 
similarities or differences. Therefore, the comparisons discussed in this report are based 
on statistical tests that consider both the magnitude of the difference between the means 
or percentages and the standard errors of those statistics. 

The statistical tests determine whether the evidence, based on the data from the 
groups in the sample , is strong enough to conclude that the averages or percentages are 
really different for those groups in the population . If the evidence is strong (i.e., the 
difference is statistically significant), the report describes the group averages or 
percentages as being different (e.g., one group performed higher than or lower than 
another group) — regardless of whether the sample averages or sample percentages 
appear to be about the same or not. If the evidence is not sufficiently strong (i.e., the 
difference is not significant), the averages or percentages are described as being not 
significantly different — again, regardless of whether the sample averages or sample 
percentages appear to be about the same or widely discrepant. Rather than relying on 
the apparent magnitude of the difference between sample averages or percentages, the 
reader is cautioned to rely on the results of the statistical tests to determine whether those 
sample differences are likely to represent actual differences between the groups in the 
population. The statistical tests and the Bonferroni procedure, which is used when more 
than two groups are being compared, are discussed in greater detail in Appendix A. 
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In addition, some of the percentages reported in the text of the report are given 
qualitative descriptions (e.g., relatively few, about half, etc.). The descriptive phrases 
used and the rules used to select them are also described in Appendix A. 

The tables in the Highlights and in Part 1 (Chapters 1 and 2) show not only the 
average scale scores for students but also the distribution of their scores at five selected 
percentiles. The distribution of the scores through these percentiles encourages the 
reader to consider the performance of the students in the various groupings (whether by 
state, region, gender, participation in federal programs, etc.) as overlapping ranges of 
heterogeneous performance, rather than as a simple monolithic average. As an example, 
consider Table 2.5 which shows that, for the nation, the 75th percentile for students 
eligible for free or reduced-price lunch is 157 while the average scale score for students 
who were not eligible for this service is 155. This means that at least 25 percent of the 
students eligible for free or reduced-price lunch performed above the average for 
students who were not eligible. 

How Is This Report Organized? 

The NAEP 1996 Science State Report for New York is a computer-generated report 
that describes the science performance of eighth-grade students in New York, the 
Northeast region, and the nation. The system to generate the state reports was developed 
because reports customized with each jurisdiction’s data would otherwise have been 
impossible to produce in a timely fashion. Because the process is automated, the 
variables reported were chosen as those most likely to be of interest to most jurisdictions. 
Unfortunately, this means that some variables of particular interest may not be reported 
here; however, each jurisdiction will receive all reportable data on CD ROM, and all 
data will be available on the NCES Web site (http://www.ed.gov/NCES/naep). Also 
because of the process, the language in the bullets and in parts of the text sometimes 
seem awkward. It is hoped that understanding the reason for these awkwardnesses will 
enable the reader to overlook them. 

A separate report describes additional eighth-grade science assessment results for 
the nation and the states, as well as the national results for grades 4 and 12. 12 This State 
Report consists of four sections: 

• This Introduction provides background information about what was 
assessed, who was sampled, and how the results are reported. 

• Part One shows the distribution of science scale score results for 
eighth-grade students in New York, the Northeast region, and the nation. 

• Part Two relates eighth-grade public school students’ science scale 
scores to contextual information about school characteristics, instruction, 
and home support for science in New York, the Northeast region, and 
the nation. In addition. Chapter 5 discusses student results of the 
hands-on tasks. 




23 

12 O’Sullivan, C.Y., C.M. Reese, and J. Mazzeo. NAEP 1996 Science Report Card for the Nation and the States . 
(Washington, DC: National Center for Education Statistics, 1997). 
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Several Appendices are presented to support the results discussed in the 
report: 



Appendix A 
Appendix B 
Appendix C 
Appendix D 



Reporting NAEP 1996 Science Results 
The NAEP 1996 Science Assessment 
Technical Appendix 
Teacher Preparation 



Other Reports of NAEP 1996 Science Results 

Related reports may be of interest to the reader: 

• Cross-State Data Compendium for the 1996 Grade 8 Science Assessment 

• Technical Report of the NAEP 1996 State Assessment Program in 
Science 

• NAEP 1996 Science Report Card for the Nation and the States 

As presently planned, there will be three additional reports appearing in late 1997 
and early 1998. One report will contain sample items and examples of student work 
on these questions. A second report will cover policy and practices in the schools and 
classrooms in the United States. A third report will cover special components of the 
NAEP science assessment, including the advanced science assessment and the hands-on 
exercises. 
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PART ONE 



Science Scale Score Results 

T he following chapters describe the average science scale scores of eighth-grade 
students in New York. As described in the Introduction, the NAEP science scale is a 
composite of the three major fields of science: earth, physical, and life. Student 
performance is generally reported on this composite scale and so reflects average student 
scores across the three fields. Student performance may also be summarized on separate 
NAEP fields of science scales that range from 0 to 300. 

This part of the report contains two chapters. Chapter 1 compares the overall 
science performance of public school students in New York to the nation. (Results for 
the Northeast region are also presented.) It also contains a U.S. map comparing the 
average scale scores in New York with other states, and a table showing students’ scale 
score distributions for the three fields of science. Chapter 2 s ummar izes science 
performance for subpopulations of public school students as defined by gender, 
race/ethnicity, parental education, participation in Title I services and programs, and 
eligibility for the free/reduced-price lunch component of the National School Lunch 
Program (NSLP). The second chapter also provides the scale score distributions for 
nonpublic school students, as well as the combined results for students in public and 
nonpublic schools. 

The NAEP 1996 assessment in science is the first developed using a new 
framework, described in Appendix B. The scale developed to report results from the 
1996 science assessment is a within-grade scale comprised of three fields of science 
scales. Appendix A describes reporting on the scale, and Appendix C describes the 
construction of the scale. 
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Item Maps 

Students’ performance is summarized on the NAEP science scale which ranges 
from 0 to 300. Nationally, public school students’ scale scores ranged from about 102 
for those scoring at the 10th percentile to about 191 for those performing at the 90th 
percentile. Sample questions are shown in Figure 1.1 illustrating the range of 
performance on the NAEP science scale for grade 8. Each question is one that is likely 
to be answered correctly by a student whose score is at or near the given percentile. 

To illustrate the range of performance in more detail, questions from the 
assessment were “mapped” onto a 0 to 300 scale, as in Figure 1.2. The item map is a 
visual representation of the scale showing selected questions in positions corresponding 
to their difficulty. The item map shows which questions a student of any particular 
ability is likely to answer correctly. The position of the question on the scale represents 
a dividing line. Students who attained scores greater than the score corresponding to the 
question’s difficulty are likely to answer it correctly, while students with scores below 
that degree of difficulty are less likely to answer it correctly. 

More specifically, students who scored below the scale score associated with a 
particular question had less than a 65 percent probability of earning a given amount of 
credit on a constructed-response question or less than a 74 percent probability of 
correctly answering a multiple-choice question. A small proportion of these students 
— those near but below the question’s position on the scale — may be more likely than 
not to answer the question correctly (between 50 and 65 or 74 percent). Such students 
are not considered “able” to answer the question, since they have not achieved sufficient 
consistency in their responses. 

This discussion and the item map illustrations refer to eighth-grade students in the 
national assessment, whose scores may not resemble those of eighth-grade students in 
New York. 
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FIGURE 1.1 


Sample Questions Likely to Be Answered Correctly by 
Grade 8 Students At or Near Selected Percentiles 




Percentile 


Question 



10th 


Find typical yearly rainfall from a graph. (104) 


25th 


Explain the impact of fish death on an ecosystem. (1 27) 


50th 


Identify the effect of acid rain. (1 50) 


75th 


Understand where earthquakes occur. (1 72) 


90th 


Explain why lightning is seen before thunder is heard. (1 94) 



The value in parentheses represents the scale score attained by students who had a 65 percent probability of reaching a 
given level on a constructed-response question (in italic type) or a 74 percent probability of correctly answering a 4-option 
multiple-choice question (in regular type). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 




26 



BEST COPY AVAILABLE 



20 



THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



New York 



Figure 1.2 is an item map for grade 8. 13 Multiple-choice questions are shown in 
regular type; constructed-response questions are in italic type. 14 An example of how to 
interpret the item map may be helpful. In this figure, a multiple-choice question 
involving interpreting a graph maps at the 136 point on the scale. This means that 
eighth-grade students with science scale scores at or above 136 are likely to answer this 
question correctly — that is, they have at least a 74 percent chance of doing so. 15 Put 
slightly differently, this question is answered correctly by at least 74 of every 100 
students scoring at or above the 1 36 scale-score level. Note that this does not mean that 
students at or above the 136 scale score always answer the question correctly or that 
students below the 136 scale score always answer it incorrectly. 

As another example, consider the constructed-response question that maps at a 
scale score of 194. This question concerns the differing speeds of light and sound. 
Scoring of responses to this question allows for partial credit by using a three-level 
scoring guide. Mapping a question at the 194 scale score indicates that at least 65 
percent of the students performing at or above this point achieved a score of 3 
(“Complete”) on the question. Among students with lower scores, less than 65 percent 
gave complete responses to the question. 




13 Details on the procedures used to develop the item map are provided in the forthcoming NAEP 1996 Technical 
Report. The procedures are similar to those used in past NAEP assessments. 

14 

The placement of constructed-response questions is based on (1) the “mapping” of a score of 3 on a 3-point scoring 
guide for short constructed-response questions and (2) the '"mapping” of a score of at least 3 on a 4-point scoring guide 
and a score of at least 4 on a 5-point scoring guide for extended constructed-response questions. 

15 For constructed-response questions, a criterion of 65 percent was used. For multiple-choice questions, the criterion 

was 74 percent. The use of a higher criterion for multiple-choice questions reflected the students' ability to “guess” 
the correct answer from among the alternatives. ,* 
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FIGURE 1.2 — GRADE 8 


Map of Selected Questions on the NAEP Science Scale 
for Grade 8 


State Assessment 



NAIP Scale 



Explain cause and prevention of aumbhng of undent monument (21 3) 

Recognize part of cell that contains genetic material (205) ► 


v ' ’i’V. 

•“ I'! !-". v :;:» 


S 

◄ (206) Know which statement is consistent with theory of evolution 


Explain changes in appearance and number of hydra (1 92) ► 
Understand forms of energy conversion (1 89) ► 


-1*2- 

(90rti prontfl* 


◄ (194) Explain why lightning is seen before thunder is heard 


Understand trend of rainfall data on graph (184) ► 




◄ (184) Understand markings of contour map to find direction of river flow 


Identify areas that have warm summers ami add winters (1 80) ► 




◄ (1 82) Understand which setup models the water cycle 


Understand where earthquakes occur (172) ► 
Know how pitch is related to length (171) ► 
Measure pH of four types of sod [] 66) ► 
Devise experiment to investigate shadow changes [\ 65) 


174 

(75 th ptwwrtth) 


◄ (172) Understand what happens when a magnet is placed inside o cod 

◄ (166) Understand movement of truck in relation to car 

◄ (163) Understand direction of movement after collision 




r 


◄ (1 58) Identify source of atmospheric oxygen 


Identify effect of add rain (150) ► 


(50th pronfil»] 


< (153) Classify organism from characteristics 


Draw m orbits on model of sohr system (\Z9) ► 




^ (148) Identify property of water that is most important for organisms 


Interpret graph showing seed production and rainfall (136) ► 


. ' " .. v . SV 


^ (135) Understand effect on density of adding more sah to solution 


Explain adnntages/disadrontoges of planting near o dream (124) ► 


(25«h pronto*) 


◄ [\V) Explain impact of fish death on ecosystem 


Interpret graph of revohrtion versus distance (121) ► 




< (121) Identify best experimental setup 


Identify organs important for oxygen transfer (113) ► 




◄ (1 14) Identify property that results from processes of living things 

< (113) Identify organisms that live in tropical rain forest 


Identify organism that produces its own food (89) ► 


— 104 — 
[10th prontO*) 


◄ (1 04) Find typical yearly rainfall from graph 

◄ (55) Determine whether markers are permanent or non-permanent 


Identify items that conduct electricity (28) ► 
2 


*'f! 


> 



NOTE: Position of questions is approximate and on appropriate scale range is displayed for grade 8. 
Italic type indicates a constracted-response question. Regular type denotes o multiple-choice question. 



Each grade 8 science question was mapped onto the NAEP 0-to-300 science scale. The position of the question on the scale 
represents the scale score attained by students who had a 65 percent probability of reaching a given score level on a constnicted- 
response question or a 74 percent probability of correctly answering a 4-option multiple-choice question. Only selected questions are 
presented. Percentiles of scale score distribution are referenced on the map. * 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 
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CHAPTER 1 



Science Scale Score Results for Eighth-Grade 
Students 

To remain competitive in the global economy, a technologically and scientifically 
literate citizenry is required. As a result, reform in science and mathematics education 
in the United States has gained increasing attention. The 1983 publication A Nation At 
Risk: The Imperative for Educational Reform called for overall reform of the United 
States educational system, with heavy emphasis placed on mathematics and science. 16 
The National Goals Panel was convened in 1989 to further focus attention on education 
reform. In 1991 the National Science Foundation’s Statewide Systemic Initiative began 
awarding grants to support state reform in K-12 mathematics and science 
instruction. 17 During the 1990s many states have been developing standards for science 
curriculum, teaching, and assessment using guidance from reform efforts such as the 
American Association for the Advancement of Science’s Project 2061, the National 
Science Teachers Association’s Scope, Sequence, and Coordination of High School 
Science, and the recently published National Research Council's National Science 
Education Standards. 18 A reaffirmation of the goal for world-class standards in 
education was made at the 1996 Governors’ Summit in Palisades, NJ. All these efforts 
address ways to produce inn ovative science curricula aimed at improving national 
scientific literacy. As a means of informing the progress of such reform, the U.S. 
Department of Education supports programs geared toward assessing the current level 
of science knowledge and skills including the Third International Mathematics and 
Science Study (TIMSS), 19 administered in 1995, and the 1996 National Assessment of 
Educational Progress (NAEP) in science. 




16 A Nation at Risk : The Imperative for Educational Reform. (Washington, DC: National Commission on Excellence in 
Education, 1983). 



Statewide Systemic Initiative. (Washington, DC: National Science Foundation, 1990). 



Science for All Americans: A Project 2061 Report on Literacy Goals in Science, Mathematics and Technology. 
(Washington, DC: American Association for the Advancement of Science, 1989); Scope, Sequence, and Coordination 
of High School Science , (Washington, DC: National Science Teachers Association, 1995); National Science Education 
Standards. (Washington DC: National Research Council, 1996). 



19 



The Third International Mathematics and Science Study was conducted in 1994 in the Southern Hemisphere and in 
1995 in the Northern Hemisphere. 
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The NAEP 1996 state science assessment at grade 8 was the first time science has 
been assessed at the state level. It continues the state-level component begun in 1990 
with the NAEP Trial State Assessment (TSA). The NAEP 1996 assessment in science 
had 47 participating jurisdictions. 20 Results for 46 jurisdictions were reported for the 
science assessment 21 

The science framework for the 1996 National Assessment of Educational 
Progress 22 was developed through a consensus process involving educators, policy 
makers, business people, assessment experts and curriculum specialists. The 1996 
NAEP science assessment included multiple-choice questions, constructed-response 
exercises, and (for the first time) hands-on tasks. Because the 1996 assessment was 
based on an essentially new framework, it is not possible to compare results from the 
1996 assessment with those from the previous NAEP science assessment in 1990. 

Table 1.1 shows the distribution of science scale scores for eighth-grade students 
attending public schools in New York, the Northeast region, and the nation. 

• The average science scale score for eighth-grade public school students 
in New York was 146. This average did not differ significantly from 
that for public school students across the nation (148). 23 
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TABLE 1.1 


Distribution of Science Scale Scores for Public School 
Students 


State Assessment 


t 





Average 
Scale Score 


10th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


90th 

Percentile 
















New York 
Northeast 
Nation 


146 ( 1.6) 
149 ( 2.9) 
148 ( 0.9) 


96 ( 2.9) 
103 ( 4.8) 
102 ( 1.6) 


122 ( 2.5) 
127 ( 4.5) 
126 ( 1.3) 


149 ( 2.0) 
151 ( 3.0) 
151 ( 0.9) 


172 ( 1.4) 

173 ( 4.0) 
172 ( 1.1) 


190 ( 1.3) 

191 ( 3.0) 
191 ( 1.3) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
sta n da r d errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



20 

Jurisdiction refers to states, territories, the District of Columbia, and the Department of Defense Education Activities 
(DoDEA) domestic and international schools. The DoDEA schools also made special arrangements to assess their 
fourth-grade students in science. 

21 

One jurisdiction did not meet minimum participation levels for public or nonpublic schools and did not have any results 
reported. 

22 

Science Framework for the 1996 National Assessment of Educational Progress. (Washington, DC: National Assessment 
Governing Board, 1993). 

23 

Differences reported as significant are statistically different at the 95 percent confidence level. This means that with 
95 percent confidence there is a real difference in the average science scale score between the two populations of 
interest. 
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Comparisons Between New York and Other Participating 
Jurisdictions 

The map on the following page shows how the average science scale score for 
eighth-grade public school students in New York compares with those of other 
jurisdictions participating in the NAEP 1996 science assessment. The different shadings 
on the map indicate whether or not the average scale scores of public school students 
in the other jurisdictions were statistically different from that of public school students 
in New York (‘Target State”). States with horizontal lines have a significantly lower 
average science scale score than New York while states with gray shading have a 
significantly higher average scale score. Unshaded states have average scale scores that 
did not differ significantly from the average for New York. States with large 
crosshatching did not meet minimum participation rate guidelines established by NCES 
for the NAEP assessments. A description of the statistical procedures used to produce 
this map is contained in Appendix A. 
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Performance in the NAEP Fields of Science 

The core of the science framework is organized along two dimensions. The first 
divides science into three major fields: earth, physical, and life. The second dimension 
defines characteristic elements of knowing and doing science: conceptual understanding, 
scientific investigation, and practical reasoning. Each question is categorized as 
measuring one of the elements of knowing and doing within one of the fields of science. 

Table 1.2 shows the distribution of scale scores for each of the three fields of 
science for New York, the Northeast region, and the nation. Appendix B describes the 
three fields of science in more detail, and Appendix C contains a discussion of the 
scaling procedures used to develop the three fields of science scales and the composite 
NAEP science scale. 

• The performance of students in New York did not differ significantly 
from that of students nationwide in the physical science, earth science, 
or life science fields described in the science framework. 



TABLE 1.2 



Distribution of Science Scale Scores for Public School 
Students by Fields of Science 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Physical Science 














New York 


146 ( 1.9) 


92 ( 5.2) 


121 (2 2 ) 


150(1.9) 


173 ( 1.7) 


193 ( 1.4) 


Northeast 


149 ( 2.2) 


101 ( 3.6) 


127 ( 3.5) 


151 ( 1.6) 


173 ( 3.8) 


193 ( 3.2) 


Nation 


149 ( 1.0) 


1 01 ( 2.0) 


126 ( 1.3) 


151 ( 1.2) 


173 ( 1.2) 


192 ( 1.6) 


Earth Science 














New York 


144 ( 1.6) 


95 ( 3.3) 


120 ( 2.6) 


147(2.1) 


171 (3.0) 


189(2.3) 


Northeast 


146 ( 3.1) 


100 ( 4.6) 


123 ( 3.5) 


147 ( 4.9) 


171 ( 3.6) 


189 ( 3.5) 


Nation 


149 ( 1.0) 


101 ( 1.9) 


126 ( 1.5) 


150 ( 1.2) 


173 ( 1.3) 


192 ( 1.9) 


Life Science 














New York 


147 ( 1.7) 


95 ( 5.1) 


122 ( 2.6) 


150(2.8) 


174 ( 2.3) 


194 ( 3.0) 


Northeast 


152 ( 3.8) 


104 (4.7) 


129 ( 3.8) 


155(4.1) 


176 ( 3.7) 


194 ( 4.4) 


Nation 


148 ( 1.1) 


100 (2.2) 


126 ( 1.3) 


151 (1.0) 


173 ( 1.1) 


191 ( 1.7) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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CHAPTER 2 



Science Scale Score Results for Eighth-Grade 
Students by Subpopulations 



The previous chapter provided a view of the overall science performance of 
eighth-grade students in New York and the nation. It is also important to examine the 
average performance of subgroups since past NAEP assessments in science, as well as 
in other academic subjects, have shown substantial differences among groups defined 
by gender, racial/ethnic background, parental education, and other demographic 
characteristics. 24 A key contribution of NAEP to the ongoing conversations concerning 
education reform is the ability to monitor the performance of subgroups of students in 
academic achievement. 

The NAEP 1996 state assessment in science provides performance information for 
subgroups of eighth graders in New York, the Northeast region, and the nation. In 
addition to the more typical demographic subgroups defined by gender, race/ethnicity, 
and parental education, the 1996 assessment also collected information on two federally 
funded programs — student participation in Title I programs and services, and student 
eligibility for the free/reduced-price school lunch program. 

The NAEP 1996 state assessment in science also continues a component first 
introduced with the NAEP 1994 state assessment in reading — assessment of a 
representative sample of nonpublic school students. The 1996 state assessment marks 
the first time that NAEP science results for public and nonpublic school students can 
be presented and compared at the state level. The comparison of public and nonpublic 
school students’ performance does not account for confounding factors such as student 
composition, family socioeconomic status, and parental involvement in their child’s 
education. The size of the NAEP nonpublic school sample in most jurisdictions does 
not allow for such in-depth analyses, and a more complete picture of public and 
nonpublic school comparisons may be achieved by supplementing NAEP results with 
data from other sources, such as the School and Staffing Survey (SASS) 25 or the 
National Education Longitudinal Study (NELS). 26 

24 Jones, L.R., I.V.S. Mullis, S.A. Raizen, I.R. Weiss, and E.A. Weston. The 1990 Science Report Card: NAEP’s 
Assessment of Fourth, Eighth , and Twelfth Graders. (Washington, DC: National Center for Education Statistics, 1992); 
Campbell, J.R., C.M. Reese, C. O'Sullivan, and J.A. Dossey. NAEP 1994 Trends in Academic Progress. (Washington, 
DC: National Center for Education Statistics, 1996). 

25 

U.S. Department of Education. Schools and Staffing in the United States: A Statistical Profile , 1993-94. (Washington, 
DC: National Center for Education Statistics, 1996). URL: http://www.ed.gov.NCES/surveys/sassJitml. 

26 National Education Longitudinal Study. National Education Longitudinal Study of 1988: Base Year Student Survey. 
(Washington, DC: National Center for Education Statistics, 1995). URL: http://www.ed.gov/NCES/suiveys/nels88.html. 
Also see a report based on NELS:88 findings, NCES 97-838: Science Proficiency and Course Taking in High School. 
This is downloadable from URL: http://www.ed.gov/NCES/pubs97/97838.html. 
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A description of the subgroups and how they are defined is presented in 
Appendix A. The reader is cautioned against making simple or causal inferences related 
to the performance of various subgroups of students or about the effectiveness of public 
and nonpublic schools or Title I programs. Average performance differences between 
two groups of students may in part be due to socioeconomic or other factors. For 
example, differences observed among racial/ethnic subgroups are almost certainly 
associated with a broad range of socioeconomic and educational factors not discussed 
in this report and possibly not addressed by the NAEP assessment program. Similarly, 
differences in performance between students eligible for Title I programs and those not 
eligible does not account for the initial performance level of the students prior to 
placement in Title I programs or differences in course content and emphasis between the 
two groups. 

Gender 

Previous NAEP results for science have shown a significant difference in the 
average scale scores of male and female eighth graders, with males having consistently 
higher scale scores. 27 As shown in Table 2.1, the NAEP 1996 state science assessment 
results for eighth graders in New York are not consistent with those general findings. 

• The average science scale score of males did not differ significantly 
from that of females in either New York or the nation. 
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TABLE 2.1 




Distribution of Science Scale Scores for Public School 
Students by Gender 


1996 
Stats Ass 


essnrei 


if 

it 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Male 

New York 
Northeast 
Nation 


148 (2.5) 

149 ( 2.5) 
149 ( 1.1) 


96 ( 4.2) 
104 ( 5.7) 
101 ( 1.8) 


122 ( 3.4) 
127 ( 4.7) 
126 ( 2.0) 


152 (2.7) 
151 ( 3.5) 

153 ( 1.1) 


176 (2.5) 
174 (2.9) 
174 ( 1.2) 


194(1.6) 
190 ( 3.6) 
192 ( 1 2) 


Female 
New York 
Northeast 
Nation 


143 ( 1.3) 
149 ( 5.2) 
148(1.2) 


96 ( 3.7) 
101 (7.4) 
103 ( 1.3) 


121 ( 2.8) 
127(8.1) 
127 ( 1.4) 


146 ( 2.8) 
151 ( 5.0) 
150 ( 1.3) 


168 ( 3.0) 
171 ( 52) 
170 ( 1.7) 


185 ( 1.3) 
192 ( 5.3) 
189 (3.4) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



27 Campbell, J.R., K.E. Voeikl, and P.L. Donahue. NAEP 1996 Trends in Academic Progress. (Washington, DC: National 
Center for Education Statistics, 1997); Jones, L.R., I.V.S. Mullis, S.A. Raizen, I.R. Weiss, and E.A. Weston. The 1990 
Science Report Card: NAEP's Assessment of Fourth, Eighth, and Twelfth Graders. (Washington, DC: National Center 
for Education Statistics, 1992). 
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Race/Ethnicity 

As part of the background questions administered with the NAEP 1996 science 
assessment, students were asked to identify the racial/ethnic subgroup that best describes 
them. The five mutually exclusive categories were White, Black, Hispanic, Asian or 
Pacific Islander, and American Indian or Alaskan Native. 

Findings from previous NAEP science assessments have shown that racial/ethnic 
differences exist in science performance. 28 However, when interpreting differences in 
subgroup performance, confounding factors related to socioeconomic status, home 
environment, and educational opportunities available to students need to be 
considered. 29 The distribution of eighth-grade science scale scores for New York, the 
Northeast region, and the nation by race/ethnicity are shown in Table 2.2. 30 



• White students in New York demonstrated an average science scale 
score that was higher than those of Black and Hispanic students but was 
not significantly different from that of Asian/Pacific Islander students. 
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TABLE 2.2 


Distribution of Science Scale Scores for Public School 
Students by Race/Ethnicity 


State Assessment 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



White 














New York 


161 ( 1.4) 


125 ( 3.1) 


143(2.1) 


162 ( 1.9) 


181 ( 1.0) 


196 ( 1.4) 


Northeast 


162 (2.6) 


128 ( 3.1) 


144 ( 4.2) 


163 ( 3.1) 


181 ( 2.4) 


198 ( 2.9) 


Nation 


159 ( 1.1) 


120 ( 1.3) 


140 ( 1 2 ) 


160 ( 1.2) 


1 79 ( 1 .2) 


196 ( 1.8) 


Black 














New York 


120 ( 1.9) 


82 ( 3.5) 


100 ( 2.2) 


120(5.8) 


139(3.0) 


156(2.7) 


Northeast 


122 ( 1.9)1 


84 ( 4.6)! 


102 ( 3.8)! 


122 (2.3)! 


141 ( 4.6)1 


160 ( 4.5)! 


Nation 


120 ( 1.2) 


81 ( 1.8) 


99 ( 1.1) 


120 ( 1.1) 


140 ( 1.6) 


158 ( 1.8) 


Hispanic 














New York 


116 ( 2.7) 


71 ( 5.3) 


91 ( 3.7) 


115 (2.8) 


144 ( 4.3) 


163 ( 6.8) 


Northeast 


132 ( 5.0)! 


91 (11.9)! 


109 ( 8.6)1 


133(7.1)! 


155 ( 4.9)! 


170 ( 6.2)! 


Nation 


127 ( 1.8) 


83 ( 3.3) 


104(2.6) 


129 ( 1.6) 


152 ( 2.7) 


170 (2.8) 


Asian/Pacific Islander 














New York 


155 ( 5.4) 


104 (16.6) 


134 ( 5.3) 


160(3.7) 


179 ( 3.9) 


195 (12.8) 


Northeast 


- <*V) 


*** j** 


~ (*V) 


*** (*V) 


- (".*) 


<*V) 


Nation 


150 ( 3.3) 


108(6.1) 


128 ( 4.4) 


151 (4.0) 


173 ( 3.3) 


190 (14.9) 



The NAEP science scale ranges from 0 to 300. Results are reported for racial/ethnic subgroups meeting established 
sample size requirements (see Appendix A). The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is wi thin ± 2 
standard errors of the estimate for the sample. In comparing two es timate s, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



Campbell, J.R., K.E. Voelkl, and P.L. Donahue. NAEP 1996 Trends in Academic Progress. (Washington, DC: National 
Center for Education Statistics, 1997); Jones, L.R., LV.S. Mullis, S.A. Raizen, LR. Weiss, and E.A. Weston. The 1990 
Science Report Card: NAEP’s Assessment of Fourth, Eighth, and Twelfth Graders. (Washington, DC: National Center 
for Education Statistics, 1992). 

29 McKenzie, F.D. “Educational Strategies for the 1990s/’ The State of Black America 1991. (New York: National Urban 
League, 1991). 

30 Results are reported for racial/e thni c subgroups meeting established sample size requirements (see Appendix A). 
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Students’ Reports of Parents’ Highest Education Level 

Students were asked to indicate the highest level of education completed by each 
parent. Four levels of education were identified: did not finish high school, graduated 
from high school, some education after high school, and graduated from college. A 
choice of “I don’t know” was also available. For this analysis, the highest education 
level reported for either parent was used. 

In general, results show that increasing parental education is associated with 
increases in student performance. In reviewing these results, it is important to note that, 
nationally, approximately 10 percent of eighth graders did not know the level of 
education that either of their parents had completed. For public school students in New 
York, this percentage was 11 percent. Despite the fact that some research has 
questioned the accuracy of student-reported data from similar groups of students, 31 past 
NAEP assessments in science, as well as other subject areas, have found that 
student-reported level of parental education exhibits a consistent positive relationship 
with student performance on the assessments. 32 Other research has corroborated NAEP 
findings. 33 

Table 2.3 shows the results for eighth-grade public school students reporting that 
neither parent graduated from high school, at least one parent graduated from high 
school, at least one parent received some education after high school, at least one parent 
graduated from college, or that they did not know their parents’ highest education level. 
The following pertains to those students who reported knowing the educational level of 
one or both parents. 

• The average science scale score of students in New York who reported 
that neither parent graduated from high school did not differ 
significantly from that of students who reported that at least one parent 
graduated from high school but was lower than that of students who 
reported that at least one parent received some education after high 
school or at least one parent graduated from college. 



31 Looker, E.D. “Accuracy of Proxy Reports of Parental Status Characteristics.” Sociology of Education, 62(4), pp. 
257-276, 1989. 

32 Jones, L.R., I.V.S. Mullis, S.A. Raizen, I.R. Weiss, and E.A. Weston. The 1990 Science Report Card: NAEP's 
Assessment of Fourth, Eighth, and Twelfth Graders. (Washington, DC: National Center for Education Statistics, 1992); 
Campbell, J.R., K.E. Voelld, and P.L. Donahue. NAEP 1996 Trends in Academic Progress. (Washington, DC: National 
Center for Education Statistics, 1997); Reese, C.M., K.E. Miller, J. Mazzeo, and J.A. Dossey. NAEP 1996 Mathematics 
Report Card. (Washington, DC: National Center for Education Statistics, 1997). 

33 National Education Longitudinal Study. National Education Longitudinal Study of 1988: Base Year Student Survey. 
(Washington, DC: National Center for Education Statistics, 1995). 
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TABLE 2.3 


Distribution of Science Scale Scores by Public School 
Students’ Reports of Parents’ Highest Education Level 


State Assessment 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Did not finish high school 














New York 


123 ( 5.4) 


81 (23.4) 


101 ( 6.6) 


121 ( 5.8) 


148 (9.4) 


167(4.7) 


Northeast 


~CV) 


*" (*V) 


(*V) 


- <*V) 


(*V) 


~ (*V) 


Nation 


131 (2.0) 


86 ( 3.0) 


108 ( 2.6) 


134 ( 4.0) 


153 ( 5.6) 


170 (3.7) 


Graduated from high school 














New York 


138 ( 3.6) 


94 ( 6.9) 


118 (5.5) 


140(3.1) 


163 (2.3) 


177 (2.1) 


Northeast 


148 (3.6) 


106 (10.0) 


128 ( 6.2) 


150 ( 6.4) 


170 (4.9) 


187 ( 7.3) 


Nation 


140 ( 1.5) 


98 ( 2.0) 


119 (2.1) 


142 ( 1.6) 


163(1.4) 


181 ( 1.2) 


Some education after HS 














New York 


147 (2.0) 


102(5.5) 


127 ( 2.6) 


151 ( 3.3) 


170 ( 2.6) 


188 ( 5.1) 


Northeast 


152 ( 3.8) 


108 (10.8) 


133 ( 6.3) 


152 ( 6.8) 


175 (7.8) 


194 (3.9) 


Nation 


155 (1.2) 


113 ( 1.0) 


137 ( 1.5) 


158(2.8) 


176(2 2) 


191 ( 1.4) 


Graduated from college 














New York 


157 ( 1.7) 


110 (2.4) 


136 (3.1) 


160 ( 1.6) 


181 (1.5) 


197(1.7) 


Northeast 


154 (4.7) 


108 (5.6) 


132 ( 4.9) 


158 ( 4.6) 


178 (3.6) 


195 (13.0) 


Nation 


157 ( 1.3) 


112 (2.1) 


137 ( 1.0) 


160 ( 1.4) 


180 (1.5) 


198 (1.3) 


1 don’t know. 














New York 


124 ( 2.4) 


78 ( 3.7) 


98 ( 2.5) 


125 ( 5.2) 


149 ( 1.8) 


167 ( 9.2) 


Northeast 


138 (10.3)! 


90 (18.3)! 


114 (7.5)! 


143 (22.7)! 


165 (7.7)! 


181 (11.4)! 


Nation 


133 (2.6) 


88 ( 3.6) 


109 ( 3.6) 


134 ( 6.5) 


157 ( 3.8) 


174 ( 4.4) 



The NAEP science scale ranges from 0 to 300. Results are reported for parental education subgroups meeting established 
sample size requirements (see Appendix A). The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
st an da r d errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 
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Title I Participation 

The Improving America’s Schools Act of 1994 (P.L. 103-382) reauthorized the 
Elementary and Secondary Education Act of 1965 (ESEA). Title I Part A of the ESEA 
provides financial assistance to local educational agencies to meet the educational needs 
of children who are failing or most at risk of failing. 34 Title I programs are designed 
to help disadvantaged students meet challenging academic performance standards. 
Through Title I, schools are assisted in improving teaching and learning and in providing 
students with opportunities to acquire the knowledge and skills outlined in their state’s 
content and performance standards. For high poverty Title I schools, all children in the 
school may benefit through participation in schoolwide programs. Title I funding 
supports state and local education reform efforts and promotes coordinating of resources 
to improve education for all students. 

NAEP first collected student-level information on participation in Title I programs 
in 1994. The NAEP program will continue to monitor the performance of Title I 
program participants in future assessments. The Title I information collected by NAEP 
refers to current participation in Title I services. Students who participated in such 
services in the past but do not currently receive services are not identified as Title I 
participants. Differences between students who receive Title I services and those who 
do not should not be viewed as an evaluation of Title I programs. Typically, Title I 
services are intended for students who score poorly on assessments. To properly 
evaluate Title I programs, the performance of students participating in such programs 
must be monitored over time and their progress must be assessed. 35 

Table 2.4 presents results for eighth-grade students by Title I participation. 

• For students receiving Title I services, the average science scale score 
of students in New York (115) was not significantly different from* that 
of students nationwide (127). The average scale score of New York 
students who were not receiving Title I services (152) was not 
significantly different from that of their national counterparts (152). 

• The average scale score of New York students who were receiving Title 
I services was lower than that of students who were not. 




* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 



34 ' ^ 

U.S. Department of Education, Office of Elementary and Secondary Compensatory Education Programs. Improving 
Basic Programs Operated by Local Education Agencies. (Washington, DC: U.S. Department of Education, 1996). 



35 



For a study of math e mati cs performance of Title I students in 1991-1992, see U.S. Department of Education, 
PROSPECTS: The Congressionally Mandated Study of Educational Growth and Opportunity, Interim Report: Language 
Minority and Limited English Proficient Students. (Washington, DC: U.S. Department of Education, 1995). 
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TABLE 2.4 


Distribution of Science Scale Scores for Public School 
Students by Title I Participation 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Participating 

New York 


115 (3.2) 


75 ( 6.0) 


95 ( 5.4) 


116 (2.0) 


136(5.0) 


155(3.7) 


Northeast 


132 (20.3)! 


88 (16.6)! 


108 (152)! 


133 (22.5)! 


157 (25.1)! 


171 (27.9)! 


Nation 


127 ( 4.9) 


82(4.1) 


102 ( 5.1) 


126 ( 5.5) 


152 ( 6.2) 


170 ( 7.4) 


Not participating 
New York 


152(1.8) 


104 ( 5.2) 


131 (3.1) 


155 ( 1.5) 


176 ( 1.4) 


193 ( 3.6) 


Northeast 


152 ( 3.7) 


107 ( 7.3) 


131 ( 5.4) 


154 (2.7) 


175 ( 3.5) 


194(4.3) 


Nation 


152(12) 


107 ( 1.8) 


131 ( 1.6) 


154 ( 1.3) 


174 ( 1.3) 


192 ( 2.3) 



The NAEP science scale ranges from 0 to 300. Results are reported for students participating in Title I programs only 
if established sample size requirements are met (see Appendix A). The standard errors of the statistics appear in 
parentheses. It can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must use the 
standard error of the difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does 
not allow accurate determination of the variability of this statistic. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



Free/Reduced-Price Lunch Program Eligibility 

The free/reduced-price lunch component of the National School Lunch Program 
(NSLP), offered through the U.S. Department of Agriculture (USDA), is designed to 
ensure that children near or below the poverty line receive nourishing meals. 36 
Eligibility for free or reduced-price meals is determined through the USDA’s Income 
Eligibility Guidelines; it is included in this report as an indicator of poverty. The 
program is available to public schools, nonprofit private schools, and residential child 
care institutions. 

NAEP first collected information on student-level eligibility for the federally 
funded NSLP in 1996. The NAEP program will continue to monitor the performance 
of these students in future assessments. 



36 U.S. General Services Administration. Catalog of Federal Domestic Assistance. (Washington, DC: Executive Office 
of the President, Office of Management and Budget, 1995). 
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Table 2.5 shows the results for eighth graders based on their participation in this 
program. 



• For students who were eligible for free or reduced-price lunch, the 
average science scale score of students in New York (124) was lower 
than that of students nationwide (133). However, the average scale 
score of students who were not eligible for this service was not 
significantly different for New York (159) than for the nation (155). 

• The average scale score of New York students who were eligible for free 
or reduced-price lunch was lower than that of students who were not. 
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TABLE 2.5 


Distribution of Science Scale Scores for Public School 
Students by Free/Reduced-Price Lunch Eligibility 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Eligible 

New York 


124 ( 1.9) 


80(2.1) 


100 ( 2.9) 


123 ( 2.3) 


150 ( 1.4) 


169 ( 1.5) 


Northeast 


138 ( 3.0) 


93 ( 3.0) 


1 14 ( 5.0) 


139 (3.5) 


162 (3.7) 


182 ( 0.8) 


Nation 


133 ( 1.7) 


87 ( 3.0) 


108 ( 2.0) 


133 ( 2.0) 


157 ( 1.7) 


176 ( 2.5) 


Not eligible 














New York 


159 ( 1.8) 


121 ( 3.1) 


141 ( 2.3) 


161 (2.6) 


1 80 ( 22 ) 


196 ( 1.6) 


Northeast 


158 ( 4.2) 


118 (3.7) 


140 ( 4.6) 


161 (4.4) 


179 ( 3.3) 


196 (2.9) 


Nation 


155 ( 1.3) 


114 ( 2.6) 


136 ( 1.4) 


157 ( 1.7) 


176 ( 1.2) 


194 (2.8) 


Information not available 














New York 


153(7.1)! 


106 (18.4)! 


131 (13.1)! 


156 ( 7.2)! 


180(6.0)! 


195 ( 6.7)! 


Northeast 


126 (24.3)1 


65 (21.5)! 


96 (22.1)! 


154 (78.4)! 


159 (31.0)! 


160 (11.9)! 


Nation 


154(3.6)! 


109 ( 6.2)! 


134 ( 5.0)! 


157 ( 2.8)! 


178(2.9)! 


196 ( 4.7)! 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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Type of School 

The NAEP 1996 state assessment marks the first time that nonpublic school 
students were assessed in science at the state level. Therefore, separate nonpublic school 
results can be reported for New York. Also, results based on a combined sample of 
public and nonpublic school students are presented. 

In 1996, approximately 84 percent of eighth graders in New York attended public 
schools, with the remaining students attending nonpublic schools (including Catholic and 
other private schools). For the nation, 89 percent of students at grade 8 attended public 
schools in 1996. 

Previous NAEP science assessments and other survey research on educational 
achievement have found significant differences in the performance of students attending 
public and nonpublic schools. 37 The reader is cautioned against using NAEP results to 
make simplistic inferences about the relative effectiveness of public and nonpublic 
schools. Average performance differences between the two types of schools may, in 
part, be related to socioeconomic and sociological factors, such as levels of parental 
involvement in their child’s education. To get a clearer picture of the differences 
between public and nonpublic schools, more in-depth investigations must be conducted 
that are beyond the scope of the NAEP state assessment program. 

Table 2.6 shows the distribution of science scale scores for the public, nonpublic, 
and combined eighth-grade populations in New York, the Northeast region, and the 
nation. 

• In New York, the average scale score of public school students (146) 
was not significantly different from that of nonpublic school students 
(149). 

• The average science scale score of students attending nonpublic schools 
in New York (149) was lower than that of nonpublic school students 
across the nation (162). 

• The average science scale score of public and nonpublic school students 
combined in New York was 146. This average was lower than that of 
students nationwide (150). 




O’Sullivan, C.Y., C.M. Reese, and J. Mazzeo. NAEP 1996 Science Report Card for the Nation and the States. 
(Washington, DC: National Center for Education Statistics, 1997); Campbell, J.R., K.E. Voelkl, and P.L. Donahue. 
NAEP 1996 Trends in Academic Progress. (Washington, DC: National Center for Education Statistics, 1997). 
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TABLE 2.6 






Distribution of Science Scale Scores by Type of School 


1996 
State Ass 


essmenl 


t 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Public 














New York 


146(1.6) 


96 ( 2.9) 


122 ( 2.5) 


149 (2.0) 


172 (1.4) 


190 ( 1.3) 


Northeast 


149 (2.9) 


103 ( 4.8) 


127 ( 4.5) 


151 (3.0) 


173 (4.0) 


191 (3.0) 


Nation 


148 (0.9) 


102 ( 1.6) 


126 ( 1.3) 


151 (0.9) 


172 ( 1.1) 


191 ( 1.3) 


Nonpublic 














New York 


149(4.7) 


112 (8.6) 


129 ( 6.3) 


148 (4.6) 


170 (6.5) 


189 (10.1) 


Northeast 


160(4.9)! 


118(5.8)! 


140(4.7)! 


163(9.1)! 


181 (2.6)! 


200 ( 3.8)! 


Nation 


162(2.5) 


123 ( 8.1) 


143 ( 3.1) 


164 ( 3.0) 


182 (2.8) 


199 (2.1) 


Combined 














New York 


146 ( 1.4) 


98 ( 2.4) 


123 ( 1.7) 


149 (1.6) 


171 ( 1 2 ) 


190 ( 1.6) 


Northeast 


151 (2.6) 


105 (3.7) 


130 ( 2.4) 


153 (4.5) 


175 (22) 


194 (4.4) 


Nation 


150(0.9) 


104 ( 1.0) 


128 ( 1.1) 


153 (0.8) 


174 ( 1.4) 


192 ( 1.4) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for die sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 
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PART TWO 



Finding a Context for Understanding 
Students’ Science Performance in Public 
Schools 

T he science performance of public school students in New York can be better 
understood when viewed in the context of the environment in which the students are 
learning. This educational environment is largely determined by school characteristics, 
by characteristics of science instruction in the school, by home support for academics 
and other home influences, and by the students’ own views about science. N AF.P 
gathers information about this environment by means of the questionnaires administered 
to principals, teachers, and students. 

Because NAEP is administered to a sample of students that is representative of the 
eighth-grade student population in the schools of New York, NAEP results provide a 
view of the educational practices in New York, useful for improving instruction and 
setting policy. However, despite the richness of the NAEP results, it is very important 
to note that NAEP data cannot establish a cause-and-effect relationship between 
educational environment and student scores on the NAEP science assessment. 

The variables contained in Part Two are from the school characteristics and 
policies questionnaire, teacher questionnaires, and student background questionnaires. 
Part Two consists of four chapters: Chapter 3 discusses school characteristics related 
to science instruction; 38 Chapter 4 describes classroom practices related to science 
instruction, including curriculum, instructional emphases, coursework, and c omp uter use; 
Chapter 5 describes portions of a hands-on task and explores student exposure to these 
experiences; and Chapter 6 covers some potential influences from the home and from 
the students’ own views about science. 

To provide additional information, the bullets below sometimes contain combined 
results from one or more categories (i.e., collapsed categories). When this is the case, 
the summed numbers reported in the bullets may be slightly different from the sums of 
the rounded numbers presented in the tables for each of the categories. 




Information on teacher preparation is included in Appendix D of this report. 
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School Science Education Policies and 
Practices 

School programs and conditions, instructional practices, and resource availability 
vary from state to state and even among schools within a locality. The information in 
this chapter is intended to give insight into those policies or practices that are associated 
with students’ success in science. 

The variables reported here reflect information from the questionnaires completed 
by principals and teachers of the public school students in the NAEP 1996 science 
assessment. In all cases, analyses are done at the student level. School and 
teacher-reported results are given in terms of the percentage of students who attend 
schools or who have teachers reporting particular practices. 39 

Emphasis on Science in the School 

In the school characteristics and policies questionnaire, principals or other head 
administrators were asked several questions relating to the priority placed on science 
within their schools. Table 3.1 presents their responses. 



• The percentage of eighth-grade students in New York who attended 
schools with a special focus on science (9 percent) was not significandy 
different from the national percentage (8 percent). 

• The percentage of eighth-grade students in New York attending schools 
that reported science was a priority (49 percent) was not significandy 
different from the national percentage (43 percent). The average scale 
score for students in these schools (138) was not significandy different 
from* that of students in schools nationwide reporting that science was 
a priority (147). 

• The average scale score of students in New York schools that reported 
that science was a priority (138) was lower than that of students in 
schools where science was not a priority (156). 

• The percentage of eighth-grade students in New York who attended 
schools that reported having a district or state curriculum that the school 
was expected to follow (93 percent) was not significandy different from 
the national percentage (94 percent). 



* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 

39 

Appendix A provides more details on the units of analysis used to derive the results presented in this report. 
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TABLE 3.1 


Public Schools’ Reports on Science as a Priority 


State Assessment 



New York 


Northeast 


Nation 


Percentage and Average Scale Score 



Is this a school with a special 
focus on science?* 

Yes 


9 ( 3.0) 


5 m 


8(2.7) 




123(4.0)! 


rn 


137(5.0)! 


Has your school identified science 
as a priority in the last two years? 
Yes 


49 ( 6.9) 


59(23.1) 


43 ( 6.8) 




138 (4.0) 


143(8.3)! 


147 ( 3.3) 


No 


51 ( 6.9) 


41 (23.1) 


57 ( 6.8) 




156 (2.8) 


- CV) 


151 ( 1.7) 


Does your district or state have a 
curriculum in science that your school 
is expected to follow?* 

Yes 


93 ( 3.3) 


100 (****) 


94 ( 2.0) 




148 (2.1) 


150(3.1) 


149 ( 1.0) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). * The response category “No” was inappropriate here because the question 
permitted several options to be selected; consequently, only “Yes” responses were tallied. ! Interpret with caution — the 
nature of the sample does not allow accurate determination of the variability of this statistic. *** Sample size is 
insufficient to permit a reliable estimate. **** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



Principals were also asked how often students received science instruction. 
Schools using block scheduling (i.e., extended periods of instruction on fewer days) were 
not separately identified. Consequently, students in schools with block scheduling who 
receive science instruction two or three times weekly may receive as many hours of 
instruction as students under traditional scheduling who receive instruction every day. 
Table 3.2 shows the following: 

• In New York, 94 percent of eighth graders attended schools that 
reported providing instruction in science every day. This percentage did 
not differ significantly from that of eighth graders across the nation 
(92 percent). 

• The average scale score for students receiving science instruction every 
day (147) was not significantly different from that of students 
nationwide receiving this much instruction (150). 
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TABLE 3.2 


Public Schools’ Reports on Time Spent in Science Instruction 


Stats Assessment 



How often does a typical 


New York 


Northeast 


Nation 


eighth-grade student in your school 
receive instruction in science? 


Percentage and Average Scale Score 



Twice a week or less/Not taught 


on 
— rv) 


2 (— *) 
— (**.*) 


0 (—). 
#*# 


Three or four times a week 


6 ( 3.1) 


5 ( ) 


8 ( 2.7) 




139 (14.3)! 


— rn 


147 ( 4.8)! 


Every day 


94 ( 3.1) 


93 (*— ) 


92 ( 2.7) 




147 (2.3) 


153 ( 3.2) 


1 50 ( 1 .2) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 



Resource Availability to Teachers 

Resources available to teachers and schools vary. Past surveys have shown that 
teachers’ perceptions of the availability of resources (e.g., materials, staff, and time) are 
variable across the country. 40 Previous NAEP assessments in other subject areas have 
shown an overall positive relationship in most states between teachers’ reports of 
resource availability and their students’ performance. 41 

Availability of Instructional Materials 

Teachers often see the lack of resources and materials as a key problem for science 
instruction. In 1993 a national survey of elementary and secondary school educators 
reported that deficiencies related to instructional resources were the most serious 
problems for science instruction in their schools. 42 In that survey, schools reported 
spending a total of $0.51 per elementary student per year and $0.88 per middle grade 
student per year on science supplies, and $50 per year on science software. (The average 
price for one piece of software is $100.) 



40 U.S. Department of Education. Schools and Staffing in the United States: A Statistical Profile, 1993-94. (Washington, 
DC: National Center for Education Statistics, 1996). 

41 For example, see Miller, K.E., J.E. Nelson, and M. Naifeh. Cross-State Data Compendium for the NAEP 1994 
Grade 4 Reading Assessment . (Washington, DC: National Center for Education Statistics, 1995); National Center for 
Education Statistics. State-by-State Background Questionnaire Data Appendix: NAEP 1992 Mathematics Assessment, 
Grades 4 and 8 . (Washington, DC: Office of Educational Research and Improvement, 1994). 

42 Weiss, I.R. A Profile of Science and Mathematics Education in the United States: 1993. (Chapel Hill, NC: Horizon 
Research, 1994). 
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Teachers whose students participated in the NAEP 1996 science assessment were 
asked to categorize how well their school systems provided them with the classroom 
instructional materials they needed. The results are shown in Table 3.3. 

• Relatively few of the students in New York had teachers who reported 
receiving all the resources they needed (11 percent). This percentage 
was not significantly different from that of students across the nation 
(11 percent). 

• The average science scale score of students in New York whose teachers 
reported receiving all the resources they needed (164) was higher than 
that of students whose teachers received some or none of the resources 
they needed (138). 



THE NATION'S 



REPORT 

CARO 


raep 


ms 




?■ 



TABLE 3.3 



Public School Teachers’ Reports on Resource Availability 



Which of the following statements is true 
about how well your school system provides 
you with the instructional materials and other 
resources you need to teach your class? 


New York 


Northeast 


Nation 


Percentage and Average Scale Score 




1 get some or none of the resources 1 need. 
1 get most of the resources 1 need. 

1 get all the resources 1 need. 


39 ( 4.4) 
138 (3.4) 

50 ( 3.6) 
153(2.7) 

11 ( 3.2) 
164 (3.3)! 


44 ( 9.9) 
141 ( 5.9)! 

41 (12.0) 

~ rn 

14 (****) 

rv) 


37(4.1) 
144 (2.0) 

52(4.1) 
153 (2.1) 
11 (3.1) 
154(5.4)! 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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Availability of Curriculum Specialist in the School 

Table 3.4 shows the percentages and average scale scores of eighth-grade students 
in public schools whose teachers indicated they had a curriculum specialist available to 
help or advise them in science. 

• In New York, about one third of the students were taught by teachers 
who reported that there was a curriculum specialist available to help or 
advise them in science (35 percent). This figure did not differ 
significantly from* that of students across the nation (43 percent). 
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TABLE 3.4 


Public School Teachers 9 Reports on Curriculum Specialists 


State Assessment 





Is there a curriculum specialist 
available to help or advise you in 
science? 


New York 


Northeast 


Nation 


Percentage and Average Scale Score 




Yes 

No 


35 ( 4.4) 
143(4.1) 
65 ( 4.4) 
152 (2.4) 


27 ( 8.2) 
- (*V) 
73 ( 8.2) 
156 ( 4.0) 


43 ( 3.9) 
148(2.7) 
57 ( 3.9) 
152 ( 1.5) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 
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Parents as Classroom Aides 

When school personnel and parents develop a positive line of communication, they 
strengthen the learning environment for the students both at school and at home. One 
of the most frequent reasons cited by school personnel for contacting parents is to 
request parent volunteer time at school. 43 The principals of the participating public 
schools were asked if parents were used as classroom aides. As shown in Table 3.5, 
principals for eighth graders reported the following: 

• Relatively few of the students in New York (9 percent) were in schools 
that reported routinely using parents as aides in classrooms while 
68 percent of students in New York attended schools where parents 
were not used as classroom aides. 
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TABLE 3.5 


Public Schools’ Reports on Parents as Aides in Classrooms 





Does your school use parents as 
aides in classrooms? 


New York 


Northeast 


Nation 


Percentage and Average Scale Score 











No 


68 ( 6.9) 


64 (19.8) 


43 ( 6.0) 




146 (2.8) 


152(52)! 


146(2.4) 


Yes, occasionally 


23 ( 6.8) 


26 (~*) 


46 ( 6.3) 




1 51 ( 6.2)! 


- r *) 


150(2.7) 


Yes, routinely 


9 ( 4.0) 


11 (7.4) 


11 (3.6) 




150 (9.5)! 




152 ( 6.9)! 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
dete rminati on of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error esti mate s cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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43 U.S. Department of Education. The Condition of Education 1995. (Washington, DC: National Center for Education 
Statistics, 1995). 



48 



THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



New York 



Student Absenteeism 

School principals were asked if student absenteeism was a serious, moderate, or 
minor problem, or not a problem. Table 3.6 shows results for eighth graders based on 
principals’ reports. 

• In New York, 13 percent of the eighth-grade public school students 
attended schools that reported that absenteeism was a moderate to 
serious problem. This percentage was not significantly different from* 
that for the nation (22 percent). 

• The average scale score of students in New York attending schools that 
reported that absenteeism was not a problem (154) was higher than that 
of students in schools where absenteeism was a moderate to serious 
problem (128). 
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TABLE 3.6 


Public Schools’ Reports on Student Absenteeism 


State Assessment 



To what degree is student 


New York 


Northeast 


Nation 


absenteeism a problem in your 
sch€>ol? 


Percentage and Average Scale Score 



Not a problem 


37 ( 5.4) 


67(11.6) 


28 ( 4.8) 




154 ( 3.5) 


- rn 


156 (3.1) 


Minor 


50 ( 5.2) 


21 (10.6) 


50 ( 4.9) 




146 ( 3.3) 


- (*V) 


149 ( 1.5) 


Moderate to serious 


13 ( 3.9) 


12 ( 7.0) 


22 ( 3.7) 




128 ( 8.8)1 


*** (-V) 


140 ( 3.0) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 
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CHAPTER 4 



Science Classroom Practices 

Science education in the nation’s schools has received considerable attention at the 
national, state, district, school, and classroom levels. In recent years, a number of 
national and international programs have measured student performance in science. The 
latest national trend report indicates that although eighth graders’ scores have shown 
recent increases, there is no significant difference in average scores between 1970 and 
1996. 44 A recent international study, the Third International Mathematics and Science 
Study (TIMSS), demonstrated that eighth-grade students’ performance in the United 
States was slightly above average compared with that of students in 40 other 
countries. 45 

Using guidance from such programs as the Statewide Systemic Initiative, Project 
Scope, Sequence, and Coordination, Benchmarks for Science Literacy, and the National 
Science Education Standards , 46 many states are currently involved in re-evaluating their 
existing standards and developing new frameworks and criteria for science instruction 
in their state. TIMSS has also pointed out some differences between classroom practices 
in the United States and in the 40 other participating nations that may guide development 
of more effective science instruction. 47 This chapter focuses on curricular and 
instructional content issues in New York public schools and their relationship to 
students’ science performance. 

For some of the issues discussed in this chapter, student- and teacher-reported 
results for similar questions are presented. In these situations, some discrepancies may 
exist between student- and teacher-reported percentages. It is not possible to offer 
conclusive reasons for these discrepancies or to determine whose reports more accurately 
reflect eighth-grade classroom activities. The results merely present students’ and 
teachers’ impressions of the science classroom. 



44 

Campbell, J.R., K.E. Voelkl, and P.L. Donahue. NAEP 1996 Trends in Academic Progress. (Washington, DC: National 
Center for Education Statistics, 1997). 




45 

Beaton, A.E., M.O. Martin, LV.S. Mullis, E.J. Gonzalez, T.A. Smith, and D.L. Kelly. Science Achievement in the 
Middle School Years: IEA 's Third International Mathematics and Science Study (TIMSS). (Chestnut Hill, MA: TIMSS 
International Study Center, 1996). 

46 

National Science Foundation, 1990, Statewide Systemic Initiative, provided grants to further research and initiatives 
in science reform; Scope ; Sequence and Coordination of High School Science . Vol.J. The Content Core: A Guide for 
Curriculum Developers. (Washington, DC: National Science Teachers Association, 1992); American Association for 
the Advancement of Science. Benchmarks for Science Literacy . (New York: Oxford University Press, 1993); National 
Research Council. National Science Education Standards . (Washington, DC: National Academy Press, 1996). 

47 

National Center for Education Statistics. Pursuing Excellence. (Washington, DC: U.S. Government Printing Office, 
1996). 
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Curriculum Coverage 

The NAEP 1996 science assessment examines three fields of science: earth, 
physical, and life. In grades 4 and 12, the 1996 NAEP framework emphasized the three 
fields of science more or less equally; however, the framework specified a heavier 
emphasis on life science at grade 8, consistent with the increasingly recognized 
importance of human biology for this age group. 48 Eighth-grade public school teachers 
were asked how much time was spent on the three traditional fields of science in then- 
classes and the results are presented in Table 4.1. 

• In New York, 38 percent of the eighth-grade public school students had 
teachers who reported spending a lot of time on earth science. This 
percentage was not significantly different than that for the nation 
(41 percent). Students in New York in classrooms where a lot of time 
was spent on earth science had an average scale score (138) that was 
lower than that of similar students nationwide (149). 

• In New York, 73 percent of the public school students had teachers who 
reported spending a lot of time on physical science. This figure was 
greater than that of their national counterparts (49 percent). The 
average science scale score in classrooms where physical sciences was 
covered a lot was not significantly different in New York (155) than 
nationwide (151). 

• In New York, 17 percent of the students had teachers who reported 
spending a lot of time on life science. This was not significantly 
different from the percentage nationwide (19 percent). The average 
scale score for students in these classrooms (142) was not significantly 
different from that of students across the nation spending a lot of timp 
on life science (147). 
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National Research Council. National Science Education Standards . (Washington, DC: National Academy Press, 1996). 
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TABLE 4.1 


-3 


; 


Public School Teachers ’ Reports on Curriculum Coverage 


State Assessment 



How much time do you spend on 
each of the following areas of 
science in this class? 



New York 


Northeast 


Nation 


Percentage and Average Scale Score 



Earth science 


None 


27 ( 5.5)t 


24 ( 6.1 )f 


7(1 .8)t 






148 (6.9)! 


~ ri 


153(4.4)! 




A little 


24 ( 5.0)t 


25 ( 9.4)t 


11 ( 3.1 )t 






160(3.4)! 


(**.*) 


153(5.6)! 




Some 


10 ( 3.6)t 


14 ( 9.0)t 


41 ( 5.0)t 






145 (6.8)! 


— rv) 


151 (2.1) 




A lot 


38 ( 5.6)t 


36 (12.6)t 


41 ( 5.6)t 






138 (4.3) 


153 ( 3.8)! 


149 (2.9) 


Physical science 


None 


4(1.5)t 


9(5.1)t 


3(1^)t 






137 (8.5)! 


— r.*) 


141 (9.5)! 




A little 


6 ( 2.0)t 


3 ( 2.2)t 


12 ( 3.6)t 






153(7.6)! 


t#t r *> 


152 (4.4)! 




Some 


17 ( 4.0)f 


17 (10.3)t 


36 ( 4.9)t 






143(6.4)! 


~ r.*) 


152 (2.8) 




A lot 


73 ( 4.6)t 


71 (10.1)t 


49 ( 4.9)t 






155(3.5) 


161 ( 5.0) 


151 ( 1.8) 


Life science 


None 


35 ( 6.3)f 


55 (15.3)t 


17(5.1)t 






147 (5.9)! 


— rn 


155 (5.0)! 




A little 


32 ( 4.8)t 


1 7 ( 8.6)t 


22 ( 4.1 )t 






1 55 ( 4.3) 


~ rn 


152 (3.5) 




Some 


16 ( 3.8)t 


14(****)t 


41 ( 6.1 )t 






152 (4.2)! 


— (*V) 


149 (2.5) 




A lot 


17 ( 5.1 )t 


13(8^)t 


19 ( 4.7)t 






142 (9.4)! 


— r.*) 


147 (2.6)! 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately dete rmin ed, -f Interpret with caution — more than 15% of the 
respondents did not answer this question. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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Eighth-Grade Students’ Course Taking 

Exposure to science and the opportunity to learn science have a positive effect on 
the science performance of students. 49 To investigate whether there is a relationship 
between science performance of students on the 1996 NAEP assessment and their study 
of science in school, information on the types of science classes in which eighth-grade 
students were enrolled and the amount of time spent each week on science instruction 
was collected. As noted for Table 3.2, in which school principals answered a similar 
question concerning the frequency of science instruction, students in schools with block 
scheduling were not identified separately. Consequently, students under block 
scheduling who receive science instruction two or three times weekly may be receiving 
as much instruction as students in traditional settings who have science eveiy day. 

Based on students’ responses shown in Table 4.2: 

• In eighth grade, 2 percent of the students in New York reported not 
taking a science course this year. This did not differ significantly from 
the national percentage (3 percent). 

• In New York, the average scale score for students taking life science 
(134) was lower than that of students taking physical science (156). 

• The average scale score for New York students taking life science (134) 
was lower than that of students taking earth science (145). 

• In New York, 84 percent of the students reported studying science three 
or more times a week. The average scale score for students who 
reported studying science three or more times a week in New York (149) 
was not significantly different from that of students studying at this level 
nationwide (152). 
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49 

Council of Chief State School Officers. State Indicators of Science and Mathematics Education . (Washington, DC: 
CCSSO, 1995). 
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TABLE 4.2 



State Assessment 



Public School Students * Reports on Their Science Classes 



New York 


Northeast 


Nation 


Percentage and Average Scale Score 



Which best describes the science 
course you are taking? 








1 am not taking science this year. 


2 ( 0.5) 


8 ( 4.4) 


3 ( 0.9) 




— r.*> 


~ ("■•) 


120 ( 3.0)! 


Life science 


13 ( 1.3) 


9(1.6) 


12(1.5) 




134 (4.4) 


126 ( 5.0)! 


133(3.5) 


Physical science 


38 ( 3.2) 


55 ( 5.5) 


25 ( 2.6) 




156 ( 3.8) 


160 ( 4.2) 


154 ( 1.6) 


Earth science 


26 ( 2.9) 


15(7.0) 


23(3.1) 




145 (2.9) 


140 ( 4.5)! 


148 ( 3.6) 


General science 


13 (1.8) 


6(1.4) 


19 (1.5) 




152 (4.4) 


*•* rv> 


156 ( 1.7) 


Integrated science 


7(1.0) 


6 ( 2.6) 


17(1.8) 




144 (4.4) 


~ rn 


156 ( 1.6) 


About how often do you study 
science in school? 






Never 


3 ( 0.5) 


6 ( 2.5) 


4 ( 0.5) 




rv> 


~ (*V) 


126 ( 3.2) 


Less than once a week 


3 ( 0.6) 


6(1.1) 


4 ( 0.3) 




r.*) 


(*V) 


136(3.0) 


1 or 2 times a week 


9 ( 0.8) 


8(1.3) 


7 ( 0.8) 




137 ( 4.0) 


- (*V) 


138 (2.6) 


3 or 4 times a week 


14 (2.4) 


18 ( 4.2) 


13 (1.9) 




140 (4.7) 


145 ( 4.7)! 


146 ( 2.2) 


Every day 


69 ( 2.9) 


62 ( 4.9) 


71 ( 2.7) 




151 ( 1.7) 


155 (4.7) 


153 (1.3) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the esti mat e for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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Instructional Emphasis 

The framework that guided the development of the NAEP 1996 science assessment 
identified three ways of knowing and doing science — conceptual understanding, 
scientific investigation, and practical reasoning. 50 In addition, the science education 
reform effort has focused heavily on students’ ability to communicate their 
understanding of science to others. 51 To assess students’ opportunities to leam and 
communicate the knowledge and skills outlined in the framework, teachers were asked 
about their plans for science instruction during the entire year. Their responses are 
shown in Table 4.3. 

• In New York, the percentage of eighth-grade students whose teachers 
reported they planned to give moderate emphasis to knowing science 
facts and terminology (53 percent) was not significantly different from* 
that of students whose teachers planned heavy emphasis on knowing 
facts and terminology (42 percent). 

• The average scale score of students whose teachers gave moderate 
emphasis to knowing facts and terminology (149) was not significantly 
different from that of students whose teachers heavily emphasized this 
topic (147). 

• Relatively few of the students in New York (10 percent) had teachers 
who reported they planned to place moderate emphasis on the 
understanding of key science concepts by their students. This 
percentage was smaller than that of students whose teachers planned 
heavy emphasis on conceptual understanding (90 percent). 

• The average scale score of students whose teachers planned moderate 
emphasis on the understanding of science concepts (146) was not 
significantly different from that of students whose teachers placed heavy 
emphasis on this topic (148). 

• In New York, the percentage of eighth-grade students whose teachers 
reported they planned to give moderate emphasis to developing science 
problem-solving skills (33 percent) was smaller than that of students 
whose teachers planned heavy emphasis on this topic (62 percent). 

• Teachers of 56 percent of the students in New York reported that they 
planned to place moderate emphasis on knowing how to communicate 
ideas in science effectively, greater than the percentage of students 
whose teachers reported giving this topic heavy emphasis (35 percent). 



* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 

50 

Science Framework for the 1996 National Assessment of Educational Progress. (Washington, DC: National Assessment 
Governing Board, 1993). 

51 American Association for the Advancement of Science. Benchmarks for Science Literacy. (New York: Oxford 
University Press, 1993); National Research Council. National Science Education Standards. (Washington, DC: National 
Academy Press, 1996). 
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TABLE 4.3 


Public School Teachers’ Reports on Instructional Emphasis 


State Assessment 



Think about your plans for your 
science instruction during the entire 
year. About how much emphasis will 


New York 


Northeast 


Nation 








you give to the following as an 
objective for your students? 


Percentage and Average Scale Score 



Knowing science facts and terminology 








Little or no emphasis 


5 ( 2.3) 


3 (™) 


5 ( 2.3) 




155 ( 8.2)! 


~ r*) 


154 ( 4.0)! 


Moderate emphasis 


53 ( 4.8) 


56 (10.0) 


57 ( 3.4) 




149 ( 3.4) 


159(4.8) 


153 (1.4) 


Heavy emphasis 


42 ( 4.5) 


42 (11.2) 


38 ( 3.9) 


Understanding key science concepts 


147(2.6) 


145 ( 6.4)! 


145 ( 2.6) 


Little or no emphasis 


1 (“") 


1 m 


0( — ) 




- (".*) 


~ fV) 


"* CV) 


Moderate emphasis 


10(2.3) 


12(6.4) 


11 (2.4) 




146(6.1)! 


— rv) 


143(2.4)! 


Heavy emphasis 


90 ( 2.5) 


86 ( 6.9) 


89 ( 2.5) 


Developing science problemsolving skills 


148(2 2) 


154(3.5) 


151 (12) 


Little or no emphasis 


4(2.3) 


4(2.2) 


3(1.6) 




153 ( 9.1)! 


— r*) 


140 (20.9)! 


Moderate emphasis 


33 ( 4.4) 


37 (12.1) 


28 ( 3.7) 




147 ( 2.8) 


rv) 


148 ( 3.4) 


Heavy emphasis 


62 ( 4.9) 


59 (12.4) 


69 ( 4.3) 


Knowing how to communicate ideas 
in science effectively 


149 ( 2.8) 


152 ( 3.3)! 


152 ( 1.3) 


Little or no emphasis 


9 ( 2.5) 


8 ( 3.2) 


16 ( 3.3) 




160(5.5)! 


~ (*V) 


151 ( 2.7)! 


Moderate emphasis 


56 ( 3.4) 


44 (12.2) 


42 ( 4.3) 




148 (2.6) 


152 ( 6.1)! 


149 (2.3) 


Heavy emphasis 


35 ( 3.6) 


48(11.5) 


42 ( 4.4) 




146(3.1) 


154(4.5)1 


151 ( 1.5) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
s tandar d errors of the es timat e for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** S tandar d error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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With the explosion of the information age, mainstream news and the Internet 
afford opportunities to access up-to-date scientific information. Science instruction 
could benefit by taking advantage of such opportunities. To determine if these 
opportunities were being explored, eighth-grade teachers and students were asked how 
often they have classroom discussions about science stories that appear in the news. 
The results are presented in Table 4.4. 

• In New York, 52 percent of eighth-grade students were taught by 
teachers who reported frequent (once a week or more) classroom 
discussions of science in the news. Relatively few of the students 
(9 percent) had teachers who reported never or hardly ever discussing 
science in the news. 

• When students were asked how often they discussed science in the news, 

31 percent reported frequent discussions while 45 percent reported 
never or hardly ever discussing it. 
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TABLE 4.4 


Public School Teachers * and Students * Reports on 
Discussions of Science in the News 



How often do your students (do you) 
discuss science in the news? 


New York 


Northeast 


Nation 


Teacher 


Student 


Teacher 


Student 


Teacher 


Student 


Percentage and Average Scale Score 



Never or hardly ever 


9 ( 2.5) 


45(1.7) 


17 (10.4) 


43 ( 3.5) 


8 ( 2.6) 


44(1.3) 




150(4.6)! 


142 ( 1.9) 


~ r*) 


145 ( 4.3) 


155 (7.6)! 


144(1.2) 


Once or twice a month 


39 ( 4.2) 


24(1.2) 


37 (12.5) 


21 (4.5) 


44 ( 4.9) 


22(1.1) 




144 ( 3.3) 


157(1.8) 


149 ( 4.4)! 


153 ( 8.6) 


150(2.1) 


155(1.9) 


Once or twice a week 


45 ( 3.5) 


21 (1.4) 


19 ( 7.9) 


22 ( 2.0) 


33 ( 2.9) 


22 ( 0.9) 




152 (2.3) 


149 ( 3.1) 


142 ( 7.9)! 


153(5.9) 


149 ( 2.0) 


154 ( 1.8) 


Almost every day 


7(1.5) 


10(1.1) 


27 (18.7) 


14 ( 5.5) 


16(4.9) 


11 (1.1) 




143(7.9)! 


142 (3.1) 


- rn 


153(7.7)! 


153(3.8)! 


147 ( 2.8) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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Science Homework 

Past NAEP science assessments have shown a positive relationship between 
science homework and performance. 52 To examine the relationship between homework 
and science scale scores in New York, the teachers of the assessed students were asked 
to report the amount of science homework they assigned each week, and students were 
asked to report the amount of time they spent on science homework each week. 

Tables 4.5 and 4.6 show the teachers’ and students’ responses for eighth-grade 
public school students in New York. (Students had an additional response choice 
“I am not taking a science course this year,” but no analogous option was available to 
teachers.) According to the teachers’ responses: 

• In New York, 0 percent of the eighth graders were not assigned science 
homework each week. In addition, 91 percent of the students were 
assigned an hour or more of homework each week. 

• The percentage of students in New York whose teachers assigned an 
hour or more of homework each week (91 percent) was not significantly 
different from the corresponding national percentage (86 percent). 
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TABLE 4.5 



Public School Teachers’ Reports on Homework in Science 



About how much time do you 
expect a student in this class to 
spend doing homework each week? 



New York 


Northeast 


Nation 


Percentage and Average Scale Score 



None 


or-) 


4(1.9) 


2 ( 0.8) 




- (-•) 


- (".*) 


134 ( 4.5)! 


1/2 hour 


9(3.1) 


13(5.3) 


12 ( 2.3) 




126(8.1)! 


134 ( 9.5)! 


142 ( 3.3)! 


1 hour 


30 ( 4.0) 


35(11.1) 


42 (4.1) 




152 ( 3.0) 


154 ( 3.5)! 


152 (2.1) 


2 hours 


39 ( 3.8) 


19 (•— ) 


28 ( 4.4) 




153(2.9) 


*** (**.*) 


152 ( 3.0) 


More than 2 hours 


21 ( 3.6) 


30 (20.3) 


15 ( 4.8) 




148 ( 4.2) 


*** (**-*) 


156(3.9)! 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



52 Jones, L.R., I.V.S. Mullis, S.A. Raizen, I.R. Weiss, and E.A; Weston. The 1990 Science Report Card: NAEP’s 
Assessment of Fourth, Eighth, and Twelfth Graders. (Washington, DC: National Center for Education Statistics, 1992). 
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The eighth-grade students’ reports indicated that: 

• Relatively few of the eighth graders did not spend any time on science 
homework in a typical week (10 percent) while 43 percent spent one 
hour or more on their science homework each week. 

• The percentage of students in New York who spent an hour or more on 
homework each week (43 percent) was greater than the percentage of 
students nationwide spending this much time on homework each week 
(34 percent). 
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TABLE 4.6 


Public School Students’ Reports on Homework in Science 


State Assessment 


i 



If you are taking science this year , 
about how much time do you spend 
doing science homework each week? 



New York 



Northeast 



Nation 



Percentage and Average Scale Score 



1 am not taking a science 
course this year. 


2 ( 0.4) 


8 ( 4.3) 


4 ( 0.9) 




(".*) 


122(3.7)! 


127 ( 3.1)! 


None 


10 ( 1.4) 


10(2.2) 


22 ( 1.5) 




145 ( 5.8) 


141 ( 7.7)! 


147 ( 1.6) 


1/2 hour 


45(1.4) 


36 ( 5.0) 


40 ( 1.4) 




147 ( 1.9) 


147 ( 3.6) 


151 ( 1.1) 


1 hour 


21 ( 0.9) 


21 ( 1.6) 


19 ( 0.7) 




145 ( 2.3) 


154 ( 5.9) 


148 ( 1.6) 


2 hours 


1 1 ( 0.9) 


14(1.6) 


8 ( 0.5) 




1 55 ( 3.7) 


163(4.4) 


156(2.7) 


3 hours 


5 ( 0.6) 


5 ( 0.6) 


3 ( 0.4) 




161 ( 3.9) 


~ r*) 


157 ( 3.1) 


More than 3 hours 


5 ( 0.7) 


6(1.5) 


4 ( 0.4) 




149 ( 5.0) 


- rv) 


152(3.5) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
sta n da r d errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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In addition to being asked about science homework in general, students were asked 
how often they use a computer at home for schoolwork. Because the question was not 
restricted to science homework, students’ reports most likely included homework for 
other academic areas such as English and mathematics. Given the trend that home 
computers are steadily assuming more importance for completing homework 
assignments, 53 it seems useful that NAEP monitor the prevalence of this practice and its 
relationship to performance. 

Based on the reports of eighth graders in New York, as shown in Table 4.7: 

• About one third of the students reported that there was no computer at 
home (35 percent) and another 13 percent reported never or hardly ever 
using their home computer to do homework. 

• Less than half of the eighth graders reported using their home computer 
to do homework almost every day (17 percent) or once or twice a week 
(19 percent). 

• The average scale score for students who used a computer almost every 
day for homework (157) was higher than that of students who never or 
hardly ever did so (140). 

• The average scale score for students who used a computer almost every 
day for homework (157) was not significantly different from that of 
students who used a computer at home once or twice a month (161). 
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TABLE 4.7 


Public School Students’ Reports on Using Computers at 


State Assessment 


Home 



How often do you use a computer 


New York 


Northeast 


Nation 


at home for schoolwork? 


Percentage and Average Scale Score 



There is no computer at home. 


35(1.5) 


41 ( 3.6) 


36 ( 1.2) 




137(1.9) 


149 ( 2.1) 


143 ( 1.0) 


Never or hardly ever 


13 ( 0.7) 


16 ( 2.8) 


17 ( 0.9) 




140 ( 2.4) 


143 ( 5.4)! 


144(1.6) 


Once or twice a month 


16 ( 1.0) 


13 ( 0.9) 


15 ( 0.5) 




161 (2.3) 


158 ( 7.5) 


160 ( 1.8) 


Once or twice a week 


19(1.1) 


15(4.3) 


17(1.1) 




156(2.3) 


153 (5.6)! 


157 ( 1.9) 


Almost every day 


17(1.3) 


15 ( 22) 


15 ( 0.7) 




157 ( 2.6) 


155 ( 4.3) 


154 ( 1.9) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 




53 U.S. Department of Education. Digest of Education Statistics 1995 . (Washington, DC: National Center for Education 
Statistics, 1995). 
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Computer Use in Science Instruction 

The use of computers in the collection of data, interpretation of results, and 
communication of findings is part of the Benchmarks for Science Literacy and the 
recently published National Science Education Standards* Recommendations for 
facilitating science instruction in the nation’s schools often include more use of 
computers. Computers can be used to demonstrate scientific concepts, simulate 
scientific phenomena, deliver instruction, and collect and analyze data. Of course, 
effective computer use may depend on many factors other than availability, such as 
teachers’ training or whether computers have been incorporated into the curriculum 
effectively. 

Computers are increasingly important in students’ homes, where they are used for 
homework as well as for other pursuits. Since 1984 the percentage of students in grades 
7 through 12 who use a computer at school or at home has increased over two-fold, to 
approximately 60 percent of students using a computer at school and 30 percent using 
one at home. 55 

Given the potential role of computers in science instruction, NAEP asked teachers 
in New York about the availability and use of computers in science instruction. As 
presented in Table 4.8, when eighth-grade science teachers in New York were asked 
about the availability of computers, their responses indicated the following: 

• In New York, 26 percent of the students were in science classes where 
computers were not available. This percentage was not significantly 
different from* that for the nation (17 percent). 

• The average scale score of New York students whose teachers reported 
not having any computers available (145) was not significantly different 
from* that of students whose teachers reported having one computer in 
the classroom (159). 



* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 

54 

Amen can Association for the Advancement of Science. Benchmarks for Science Literacy. (New York: Oxford 
University Press, 1993); National Research Council. National Science Education Standards. (Washington, DC: National 
Academy Press, 1996). 

55 U.S. Department of Education. Digest of Education Statistics 1995. (Washington, DC: National Center for Education 
Statistics, 1995). 
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TABLE 4.8 



Public School Teachers’ Reports on the Availability of 
Computers 



New York 


Northeast 


Nation 


Percentage and Average Scale Score 



Which best describes the 
availability of computers for use by 
your science students? 



None available 


26 ( 4.6) 


4(1.6) 


17 ( 3.4) 




145 ( 4.5)! 


— rv) 


149 ( 5.6)! 


One within the classroom 


10(2.9) 


12 (*"*) 


22 ( 4.8) 




159 (6.1)! 


— rv> 


149 ( 3.2)! 


Two or three within the classroom 


6 ( 2.6) 


22 (****) 


9 ( 4.6) 




152 (5.4)! 


— (**.*) 


156 ( 7.2)! 


Four or more within the classroom 


4(2.1) 


om 


7 ( 3.0) 


Available in 8 computer laboratory 


- (*V) 


— (**.•) 


159 ( 2.8)! 


but difficult to access or schedule 


35 ( 4.8) 


50 (12.0) 


32 ( 4.9) 


Available in a computer laboratory 


147 (4.1) 


145 ( 5.9)! 


149 ( 2.1) 


and easy to access or schedule 


19 ( 3.5) 


12 ( 5.5) 


13 ( 2.6) 




146 ( 5.2) 


(**.*) 


148 ( 2.4) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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The availability of computers varies from school to school, and the uses for 
computers can vary widely from class to class. Computers can be used in many ways 
to help students learn science, including simulating scientific phenomena or illustrating 
models. Also, the frequency of use can vary, regardless of the primary use in the 
classroom. Teachers in New York were asked how they used computers and how often 
they were used in their science classrooms. Also, students were asked how often they 
used computers when doing science in school. The responses of eighth-grade public 
school teachers to the purpose of use for science instruction, as shown in Table 4.9, 
indicate the following: 

• The percentage of New York students whose teachers reported that they 
used computers for simulations and modeling (12 percent) was smaller 
than the corresponding national percentage (26 percent). 

• The percentage of students in New York whose teachers reported that 
their use of computers for instruction in science was for data analysis 
and other applications (20 percent) was not significantly different from 
that of students nationwide (20 percent). 

• About half of the eighth graders had teachers who reported not using a 
computer for science instruction (51 percent). This percentage did not 
differ significantly from the percentage for the nation (46 percent). 

Table 4.10 presents teacher and student reports on the frequency of use of 
computers for science. 

• In New York, 67 percent of the students had teachers who reported 
never or hardly ever using a computer with their classes, while a small 
percentage reported doing so almost every day (1 percent) or once or 
twice a week (6 percent). 

• In New York, 69 percent of the students reported never or hardly ever 
using computers to do science in school. Furthermore, 5 percent of the 
students reported using computers almost every day and 8 percent used 
them once or twice a week. 




64 



THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



New York 



THE 1 
REPORT 
CARD 

“BOB 


jjOTON; 

raq 

& 


s 

i 


TABLE 4.9 


Public School Teachers’ Reports on the Use of Computers for 


State Assessment 


Instruction in Science 





How do you use computers for 
instruction in science? 


New York 


Northeast 


Nation 


Percentage and Average Scale Score 











Drill and practice 


11 (2.4) 


22 n 


8 ( 4.4) 




149 (5.1)! 


- rv) 


155(6.8)! 


Playing science/teaming games 


12 ( 3.0) 


23 (12.1) 


20 ( 3.8) 




145 ( 4.5)! 


~ rv) 


150 ( 3.9) 


Simulations and modeling 


12(3.1) 


31 (19.2) 


26 ( 5.5) 




148 ( 7.7)! 


- rv) 


153(2.4)! 


Data analysis and other applications 


20 ( 4.3) 


14(6.1) 


20 ( 3.5) 




153 ( 3.3)! 


*** (**.*) 


149 ( 1.6) 


Word processing 


23 ( 3.6) 


21 (11.9) 


22 ( 3.5) 


1 do not use computers for 


148 ( 3.4) 


^ (**.*) 


152 ( 2.2) 


science instruction. 


51 ( 4.0) 


35(9.2) 


46(4.2) 




148 (2.9) 


145 ( 5.9)! 


149(2.1) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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TABLE 4.10 


Public School Teachers’ and Students’ Reports on the 


State Assessment 


Frequency of Computer Use 



How often do your students (do you) 
use a computer for science? 


New York 


Northeast 


Nation 


Teacher 


Student 


Teacher 


Student 


Teacher 


Student 


Percentage and Average Scale Score 



Never or hardly ever 


67 ( 4.5) 
150 (2.3) 


69 (2.1) 
148 ( 1.8) 


37 ( 9.0) 
144 ( 5.8)! 


68 ( 4.4) 
150 (2.7) 


62 ( 4.3) 
150 ( 1.8) 


67 ( 1 .8) 
150 ( 1.1) 


Once or twice a month 


27 ( 4.6) 
144 ( 5.5)! 


18 ( 1.6) 
151 (2.9^ 


59 (10.3) 
157(5.2)! 


16 ( 3.8) 
156(7.4)! 


31 ( 4.0) 
151 ( 2.2) 


18(1.1) 
154 ( 1.9) 


Once or twice a week 


6(1.7) 

154(8.7)! 


8 ( 0.8) 
142(4.6) 


4H 

*** r.*> 


10 ( 2.7) 
148 (10.6)! 


7 ( 2.4) 
156(4.0)! 


10 ( 1.0) 
145(2.9) 


Almost every day 


1 (****) 
*** 


5 ( 0.7) 
143(5.4) 


0 (“) 
~ rv) 


7 ( 2.0) 

— rv> 


0 ( 0.3) 


5 ( 0.5) 
135 ( 3.6) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



6 ? 



65 



BEST COPY AVAILABLE 



New York 



CHAPTER 5 



Student Performance on Hands-On Science 
Tasks 

A number of goals for science education have been put forward in a series of 
reports authored by government agencies and professional societies over the last 15 
years. 56 These goals include acquisition of a core of scientific understanding, ability to 
apply science knowledge in practical ways, familiarity with experimental design, and the 
ability to carry out scientific experiments. The reports also offered recommendations 
for the science curricula and instruction needed to achieve these goals, such as 
encouraging active student participation in hands-on science, incorporating cooperative 
group learning, and assignment of sustained projects to students. 57 

A 1993 national survey indicated that science teachers devote 21 to 26 percent 
of class time to hands-on or manipulative activities. 58 While research on the relationship 
between exposure to hands-on science tasks and overall science performance is sparse 
and inconclusive, a recent study has demonstrated a positive relationship for 
eighth-grade students between the frequency of hands-on activities and their performance 
on a standardized assessment. 59 




56 National Science Board Commission on Precollege Education in Mathematics, Science, and Technology. Educating 
America for the 21st Century. (Washington, DC: National Science Foundation, 1983); Science for All Americans: A 
Project 2061 Report on Literacy Goals in Science, Mathematics , and Technology. (Washington, DC: American 
Association for the Advancement of Science, 1989); Aldridge, B.G. Essential Changes in Secondary School Science: 
Scope, Sequence, and Coordination. (Washington, DC: National Science Teachers Association, 1989); National 
Research Council. Fulfilling the Promise: Biology Education in the Nation’s Schools. (Washington, DC: National 
Academy Press, 1990). 

57 

Science Framework for the 1996 National Assessment of Educational Progress. (Washington, DC: National Assessment 
Governing Board, 1993). 

58 

Blank, R.K. and D. Gruebel. State Indicators of Science and Mathematics Education. (Washington, DC: Council of 
Chief State School Officers, 1995). 

59 Stohr-Hunt, P.M. “An Analysis of Frequency of Hands-On Experience and Science Achievement” Journal of Research 
on Science Teaching, 33. (1996, pp. 101-109). 
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NAEP included assessments of higher-order thinking skills in science and 
mathematics as early as 1986 through a pilot assessment that required students to work 
on various hands-on tasks. Although the NAEP 1990 science assessment measured 
skills that were integral to scientific investigation, 60 hands-on tasks were not included. 
When the 1996 science framework 61 was developed in the early 1990s, it took into 
account the current reforms in science education by specifying three question types that 
probed understanding of conceptual and reasoning skills: performance exercises, 
constructed-response questions, and multiple-choice questions. It was envisaged that in 
the performance exercises, students would manipulate selected physical objects and try 
to solve a scientific problem using the objects before them. Hands-on tasks that met 
these criteria were developed for the 1996 science assessment, and each student who 
participated in the assessment was given an opportunity to conduct one of them. 

NAEP Hands-On Science Tasks 

Four different hands-on tasks were administered in the NAEP 1996 science 
assessment. Each task was designed to use materials to perform an investigation, make 
observations, evaluate experimental results, and apply problem-solving skills. In 
addition, tasks shared the following characteristics: 

• Diagrams were included to guide students through the procedures; 

• Multiple-choice and constructed-response questions were embedded 
throughout the tasks; and 

• Scientific investigation was integrated with conceptual understanding 
and practical reasoning. 

The creation of the hands-on tasks presented special challenges. Since the 
assessment was administered in a variety of settings, ranging from laboratories to 
cafeterias, all of the required equipment necessary to conduct each task had to be 
provided in a self-contained kit produced according to standard specifications to ensure 
uniformity. There were some limitations on materials and equipment. For example, live 
materials (with the exception of seeds) and equipment that required an electric outlet 
were not used. Safety was also an important concern and was addressed in a number 
of ways. The state’s safety regulations were considered; no toxic or corrosive chemicals 
were used; assessment administrators were trained in appropriate laboratory safety; and 
students were provided with goggles for some tasks. 

A brief summary of one of the four hands-on tasks is described in this chapter. 
Several questions from the hands-on task are also shown with their scoring criteria. 




60 Science Objectives : 1990 Assessment. (Princeton, NJ: The National Assessment of Educational Progress, 1989). 

61 Science Framework for the 1996 National Assessment of Educational Progress. (Washington, DC: National Assessment 
Governing Board, 1993). 
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Sample Questions from a Task 



A brief summary of one of the four tasks given to grade 8 students in New York 
is presented below with sample questions in Figures 5.1 and 5.2. 

Salt Solutions: Estimating the Salt Concentration of an Unknown Salt Solution 
Using the ‘^Floating Pencil Test” 

An instrument constructed from a pencil and thumbtack served as a hydrometer in this 
task. Students were asked to observe, measure, and compare the lengths of a portion 
of the pencil, marked with calibrations for ease of measurement, that floated above 
the surface in distilled water and in a 25 percent salt solution. Based on these 
observations, students were asked to predict how the addition of more salt to the salt 
solution would affect the floating pencil: Students then measured the length of the 
pencil that floated above the surface of a solution of unknown salt concentration and 
used the results of their previous observations to estimate the salt concentration of the 
unknown solution. The task assessed students’ ability to make simple observations, 
measure length using a ruler, apply observations to an unknown, draw a graph, 
interpolate from graphical data, and make a generalized inference from observations. 

The task also assessed students’ understanding of the value of performing multiple 
trials of the same procedure. 

Figure 5.1 shows a data table that was presented in the first stage of the task. Questions 
3, 4, and 5 are also presented in this figure. Students were asked to measure the length 
of pencil floating above the surface in three solutions: distilled water, a 25 percent salt 
solution, and a solution containing an unknown concentration of salt. The students 
recorded two measurements for each of the 3 solutions in Table 1 and calculated the 
average of each pair of readings. The scoring rubrics for Complete responses are shown 
in Figure 5.1. 
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Salt Solutions Task: Questions 3, 4, and 5 





3. Now take the pencil out of the water and dry it with a paper towel. 
Use the ruler to measure the length of the pencil that was above the 
water. Record the length in Table 1 below under Measurement 1. 



TABLE 1 





Type of 
Solution 


Length of Pencil Above Water Surface (cm) 


Measurement 

1 


Measurement 

2 


Average 




Distilled Water 










Salt Solution 










Unknown Salt 
Solution 











4. Now place the pencil back in the distilled water and repeat steps 2 and 3. 
Record your measurement in Table 1 under Measurement 2. 

5. Calculate the average of Measurements 1 and 2 and record the result 
in the data table. 

(You can calculate the average by adding Measurement 1 + Measurement 2 
and then dividing by two.) 

SCORING RUBRIC 

Measurement: A Complete response has three pairs of measurements that agree 
within a given tolerance and also are in the correct relative order. 

Average: A Complete response correctly calculates the average for each set of data. 
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Students were then presented with graph paper and asked to plot the average of 
the measurements for distilled water and 25 percent salt solution against salt 
concentration. They were told to assume a linear relationship between the height of the 
pencil above the solution and the salt concentration, and then asked to use the graph to 
determine the salt concentration of the unknown solution (Figure 5.2). The scoring 
rubric for a Complete response is also shown. 
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Salt Solutions Task: Question 14 


State Assessment 



14. Based on the graph that you plotted, what is the salt concentration of 

the unknown solution? 

Explain how you determined your answer. 



SCORING RUBRIC 

Unknown Solution: A Complete response gave a salt concentration consistent with 
the graph and correctly explained how the graph was used to obtain the answer. 



Instruction Related to Scientific Investigation 

Research devoted to the effectiveness of hands-on tasks is ongoing, although there 
is evidence that eighth graders who are exposed to hands-on activities more frequently 
perform better on standardized assessments. 62 Eighth-grade science teachers in New 
York were asked about the emphasis they placed on laboratory skill s and data analysis 
in their science classes and about the frequency and nature of hands-on activities or 
investigations assigned by them. Students were asked about the frequency and nature 
of hands-on activities or investigations conducted by them. 

As mentioned before, a direct cause-and-effect relationship between educational 
environment and student scores on the NAEP science assessment is not implied. For 
instance, the motivation and expectations of teachers or students reporting hands-on 
investigations hardly ever or once or twice a week may be a factor in the average score 
differences. However, responses to teacher (and school) questionnaires provide a broad 
view of educational practices that should prove useful for improving instruction and 
setting policy. 




62 Stohr-Hunt, P.M. “An Analysis of Frequency of Hands-On Experience and Science Achievement” Journal of Research 
on Science Teaching, 33. (1996, pp, 101-109). 
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Teachers’ and students’ responses regarding scientific investigation are presented 
in Tables 5.1 through 5.5. 

• The percentage of eighth-grade students in New York whose teachers 
reported placing heavy emphasis on the development of laboratory skills 
and techniques (47 percent) was not significantly different from the 
percentage nationwide (42 percent). Students whose teachers reported 
heavy emphasis on laboratory skills and techniques in New York had 
an average scale score (153) which did not differ significantly from that 
of students nationwide whose teachers reported this (153). 

• The percentage of eighth-grade students in New York whose teachers 
reported moderate to heavy emphasis on the development of data 
analysis skills (88 percent) was not significantly different from that of 
students nationwide (89 percent). Eighth-grade students whose teachers 
reported moderate to heavy emphasis on data analysis skills had an 
average science scale score (149) which did not differ significantly from 
that of students whose teachers reported little or no emphasis on the 
development of data analysis skills (147). 
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TABLE 5.1 


Public School Teachers’ Reports on Science Instruction 
Related to Performance Tasks 


State Assessment 


t 



Think about your plans for your science 
instruction during the entire year. About how 
much emphasis will you give to each of the 
following? 



New York 


Northeast 


Nation 


Percentage and Average Scale Score 



Developing laboratory skills and techniques 
as an objective for your students 
Little or no emphasis 


10(2.6) 
133 (7.8)! 


15 ( 3.6) 
139 ( 8.4)! 


13 (2.5) 
135 ( 3.6)! 


Moderate emphasis 


• 42 ( 4.3) 
147 ( 2.9) 


46 (14.0) 
153 ( 9.1)! 


44 ( 4.7) 
152 ( 2.0) 


Heavy emphasis 


47 ( 4.5) 
153(2.5) 


39 (12.7) 


42 ( 4.5) 
153 (2.1) 


Developing data analysis skills 
Little or no emphasis 


12(2.9) 
147 ( 6.4)! 


10 (4.6) 
139 ( 5.1)1 


11 (2.7) 
139 ( 5.5)! 


Moderate emphasis 


56 ( 5.2) 
146 ( 3.4) 


64 (22.6) 
153 (5.1)! 


65 ( 5.3) 
151 ( 1.6) 


Heavy emphasis 


32 ( 4.7) 
153(3.5) 


26 (—* ) 
r*) 


24 ( 4.3) 
153 ( 3.0) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 




72 



73 

THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



New York 



• About two thirds of the eighth-grade students in New York (69 percent) 
had teachers who reported doing a science demonstration at least once 
a week, not significantly different from* the percentage of students 
nationwide (59 percent). Less than half of eighth-grade students in New 
York (47 percent) reported that their teacher performed science 
demonstrations at least once a week. 

• The eighth-grade students in New York whose teachers reported doing 
a science demonstration at least once a week had an average scale score 
(148) which did not differ significantly from that of their national 
counterparts (151). 
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TABLE 5.2 


Public School Teachers’ and Students’ Reports on the 
Frequency of Science Demonstrations 


State Assessment 



How often do you (does your teacher) 
do a science demonstration? 


New York 


Northeast 


Nation 


Teacher 


Student 


Teacher 


Student 


Teacher 


Student 


Percentage and Average Scale Score 



Never or hardly ever 


3(1.3) 
^ (-.•) 


27 ( 1.6) 
137 ( 2.3) 


3 (™) 

~ rn 


28 ( 3.0) 
141 ( 3.1) 


2 ( 0.8) 
149 (11.6)! 


30 ( 1.3) 
141 ( 1.5) 


Once or twice a month 


27 ( 4.9) 
150 ( 3.9)! 


26 ( 1.2) 
150 ( 2.3) 


16(7.1) 

- r.*) 


28 ( 3.5) 
152 (2.6) 


39 ( 4.1) 
150 ( 2.0) 


29(1.1) 
151 ( 1.3) 


Once or twice a week 


55 ( 5.5) 
146 ( 2.6) 


29(1.7) 
157 (2.1) 


73 ( 5.6) 
155 ( 5.0) 


33 ( 4.5) 
156 ( 4.1) 


49 ( 3.5) 
152 ( 1.9) 


28 ( 1.2) 
156 ( 1.4) 


Almost every day 


14 ( 3.8) 
158 ( 4.6)! 


18 ( 1.6) 
148 (3.1) 


8 ( 4.6) 
^ CV) 


12 ( 1.9) 
153 ( 6.9)! 


10(2.3) 
144 ( 2.0)! 


14 ( 0.9) 
153(2.0) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 




* Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 
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• The percentage of eighth-grade students in New York whose teachers 
reported their science students performed hands-on tasks once a week 
or more (70 percent) was smaller than the national percentage 
(83 percent). The percentage of students in New York whose teachers 
reported their students never or hardly ever did hands-on tasks 
(3 percent) was not significantly different from nationwide levels 
(1 percent). 

• The eighth-grade students in New York whose teachers reported their 
students did hands-on tasks at least once a week had an average science 
scale score (153) which did not differ significantly from that of students 
nationwide whose teachers reported this same level of hands-on task 
experience (153). 

• The eighth-grade students in New York whose teachers reported their 
students did hands-on tasks almost every day had an average scale score 
(156) which was higher than that of students whose teachers reported 
doing hands-on activities once or twice a month (141). 
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TABLE 5.3 


Public School Teachers’ and Students’ Reports on the 
Frequency of Hands-on Activities or Investigations 




How often do your students (do you) 
do hands-on activities or 
investigations in science? 


New York 


Northeast 


Nation 


Teacher Student 


Teacher Student 


Teacher Student 


Percentage and Average Scale Score 



Never or hardly ever 


3(1.2) 
116 (4.6)! 


20(1.6) 
128 ( 2.6) 


2 ("**) 
(**.*) 


15 ( 1.6) 
132 (3.5) 


1 ( 0.6) 
119 (4.0)! 


18(1.1) 

134(1.2) 


Once or twice a month 


27 ( 5.4) 
141 (4.3)1 


32 ( 1.8) 
147(2 2 ) 


14 (3.9) 

••• ftj 


30 ( 1.2) 
153(2.6) 


16(2.4) 
140 ( 3.4) 


32(1.5) 

152(1.5) 


Once or twice a week 


55 ( 5.8) 
152 (3.0) 


31 ( 1.9) 
158 ( 2.0) 


76(5.1) 
156 (3.9) 


38 ( 2.7) 
157 ( 3.4) 


64 ( 3.5) 
153 ( 1.5) 


33(1.3) 
155 ( 1 2 ) 


Almost every day 


15 ( 3.8) 
156 (4.6)! 


17(2 2 ) 
152 ( 3.3) 


8(5.2) 

r •*) 


17 ( 2.3) 
147 ( 4.7) 


19(3.2) 
152 ( 22 ) 


18(1.1) 
151 ( 1.5) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 
**** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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• About three quarters of the eighth-grade students in New York 
(77 percent) had teachers who reported assigning science projects in 
school which take a week or more to complete. More than half of the 
students (60 percent) reported receiving such assignments and their 
average scale score was 148. 

• The average scale score of students who reported doing science projects 
or investigations that take a week or more (148) was not significantly 
different from that of students who did not (146). 
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TABLE 5.4 


Public School Teachers’ and Students’ Reports on Lang-Term 
Science Projects 


State Assessment 



Do you ever assign (do) individual 
or group science projects or 
investigations in school that take a 
week or more? 


New York 


Northeast 


Nation 


Teacher 


Student 


Teacher 


Student 


Teacher 


Student 


Percentage and Average Scale Score 



Yes 


77 ( 4.5) 
149 (2.3) 


60 ( 2.5) 
148 (2.0) 


80 ( 7.3) 
153(4.4) 


60 (12.7) 
149 ( 5.0)! 


82 ( 2.6) 
151 (1.3) 


63 ( 2.8) 
151 ( 1.3) 


No 


23 ( 4.5) 
151 (6.1)! 


40 ( 2.5) 
146(2.4) 


20 ( 7.3) 
150 (6.3)! 


40 (12.7) 
151 (4.8)! 


18 ( 2.6) 
147 ( 3.4) 


37 ( 2.8) 
146 ( 1.7) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for die entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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• In New York, the eighth-grade students who reported designing and 
carrying out their own scientific investigations once a week or more 
frequently (13 percent) received an average scale score of 134. 

• The average scale score for New York students who reported designing 
and carrying out their own science investigations once a week or more 
(134) was lower than that for students who reported doing this once or 
twice a month (150). 
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TABLE 5.5 


Public School Students’ Reports on Independent Science 
Investigations 


State Assessment 


t 



When you study science in school, how often 
do you design and cany out your own 
science investigations? 



New York 


Northeast 


Nation 


Percentage and Average Scale Score 



Never or hardly ever 


67 ( 1.9) 
150 ( 1.4) 


64 ( 1.6) 
153 ( 3.4) 


63(1.1) 
151 ( 1.0) 


Once or twice a month 


20 ( 1.5) 
150(2.9) 


21 (1.7) 
148 ( 3.8) 


23 ( 0.8) 
151 ( 1.3) 


Once or twice a week 


9 ( 0.7) 
132(3.9) 


10(2.1) 
138 ( 7.6)! 


10 ( 0.6) 
142 ( 2.3) 


Almost every day 


5 ( 0.6) 
137 ( 5.6) 


5 ( 0.9) 
- <*V) 


5 ( 0.4) 
137 ( 2.5) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). ! Interpret with caution — the nature of the sample does not allow accurate 
determination of the variability of this statistic. *** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 



77 

ERjfc 



76 



THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



New York 



CHAPTER 6 



Influences Beyond School that Facilitate 
Learning Science 

The home environment can be an important support for the school environment. 
To examine the relationship between science scale scores and home factors, data 
regarding students’ responses to questions about home factors and principals’ responses 
to questions about parental involvement in the school were examined. The student 
questionnaires also asked students how often they had changed schools because of 
household moves to examine the impact of student mobility on academic achievement. 

Students’ attitudes toward science can influence their performance in the 
assessment. For example, in a recent large scale science assessment, students who 
agreed that science learning is useful for the future and that science should be required 
in school performed better than those who disagreed with these statements . 63 These 
attitudes toward science may be attributed to factors within the school and external 
influences. The beliefs and general impressions that secondary school students form 
about science can affect not only their performance in assessments but also their 
decisions about pursuing scientific careers in the future . 64 




63 Campbell, J.R., C.M. Reese, C. O’Sullivan, and J.A. Dossey. NAEP 1994 Trends in Academic Progress. (Washington, 
DC: National Center for Education Statistics, 1996). 

64 

Gallagher, S A. “Middle School Classroom Predictors of Science Persistence.” Journal of Research in Science 
Teaching, 1994, 33. pp. 721-734. 
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Discussing Studies at Home 

The importance of schoolwork for students and their families can by measured 
by how often it is discussed at home. When students discuss academic work at home, 
they create an important link between home and school. Recent NAEP assessments in 
various subject areas have found a positive relationship between discussing studies at 
home and student performance. 65 

The NAEP 1996 assessment asked students to report on how frequently they 
discuss schoolwork at home. As shown in Table 6.1, the results for eighth graders 
attending public schools in New York indicate that: 

• Less than half of the eighth graders (44 percent) said they discussed 
their schoolwork at home almost every day. This percentage was greater 
than the percentage who said they never or hardly ever had such 
discussions (20 percent). 

• The average scale score for students who discussed their schoolwork 
almost every day (151) was higher than that for students who never or 
hardly ever did so (143). 
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TABLE 6.1 


Public School Students’ Reports on Discussing Studies at 
Home 



How often do you discuss things 


New York 


Northeast 


Nation 


you have studied in school with 
someone at home? 


Percentage and Average Scale Score 



Never or hardly ever 


20 ( 1.1) 


22(2.1) 


21 ( 0.8) 




143 ( 1.9) 


146 ( 2.4) 


141 ( 1.5) 


Once or twice a month 


9 (0.7) 


10(1.1) 


9 ( 0.4) 




142 (4.0) 


158 ( 4.4) 


149 ( 1.6) 


Once or twice a week 


27 ( 1.2) 


24 ( 2.5) 


28 ( 1.0) 




150 ( 2.3) 


147 ( 6.0) 


151 ( 1.3) 


Almost every day 


44 ( 1.7) 


44(32) 


41 (1.1) 




151 ( 1.8) 


153 ( 2.5) 


153 ( 12) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment 
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Literacy Materials in the Home 

Students can learn much about science by reading materials outside the classroom. 
For example, scientific information can often be found in mainstream newspaper and 
magazine articles. Also, the availability of reading and reference materials at home may 
be an indicator of the value placed on learning by the parents. 1 In recent NAEP 
assessments, a positive relationship has been reported between print materials in the 
home and average scale scores. 2 

The NAEP science assessment asked students whether their families had more than 
25 books, an encyclopedia, a newspaper, or any magazines in their home. Table 6.2 
shows the percentages of eighth-grade public school students reporting that their families 
have all four types, only three types, or two or fewer types of these literacy materials. 
The table also presents students’ corresponding average scale scores. Based on their 
responses: 

• About half of the students in New York (51 percent) reported having 
all four types of literacy materials in their homes. This percentage was 
somewhat greater than the percentage for the nation (47 percent). 

• The percentage of students in New York reporting having two or fewer 
types of these materials (23 percent) was smaller than the percentage 
having all four types (5 1 percent). The percentage having two or fewer 
types did not differ significantly from the percentage for the nation 
(24 percent). 

• The average science scale score for students in New York with all four 
types of literacy materials (156) was higher than that for students with 
two or fewer types (125). 
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